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ABSTRACT 

The  n.sp  and  acceptance  by  Navy  personnel  of  16  anjor 
training  devices  w*/ve  studied  in  relation  to;  {!.)  situa¬ 
tional  factors  affecting  training;  (2)  simulation  charac¬ 
teristics  of  the  trainer;  (3)  instructional  characteristics 
of  the  trainers;  reliability  of  the  trainers;  formal  end 
Informal  communications  regarding  trainer  capabilities;  and 
level  of  experience  of  the  users  in  the  systems  simulated 
by  the  trainers.  Trainers  representing  air,  surface,  and 
submarine  systems  *ere  selected  for  study;  the  participants 
included  students,  instructors,  administrative,  and  main¬ 
tenance  personnel. 

An  acceptance  profiling  technique  wns  developed  that 
appears  to  be  highly  diagnostic  of  the  reasons  for  accep¬ 
tance  or  rejection  of  particular  trainers.  It  was  evident 
that  both  highly  accepted  and  seriously  rejected  trainers 
were  represented  in  the  sample.  Methods  for  increasing 
trainee  acceptance  are  outlined  in  terms  of  improvement  in 
specific  areas  of  simulation;  improved  software;  greater 
qualifications  for  instructors;  improved  evaluation  of  per¬ 
formance;  and  improved  understanding  of  the  purpose,  capa¬ 
bilities,  and  limitations  of  trainers  by  the  users.  The 
merits  of  continuing  studies  of  trainer  acceptance  and  the 
role  of  a  ,vtTainer  advocate”  are  discussed. 
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FOREWORD 

An  important  determinant  of  the  effectiveness  of  training  device 
systems— one  that  has  received  relatively^ little  research  attention — 
is  the  attitude  of  the  trainees,  instructors,  and  managers  toward 
them. 

This  study  was  undertaken  to  identify  the  factors  leading  to  the 
acceptance  or  rejection  of  training  devices  by  their  users  and  to 
determine  the  extent  to  which  these  factors  were  present  in 
"effective"  and  "ineffective"  training  devices.  Future  training 
devices,  including  the  associated  training  materials  and  recommenda¬ 
tions  for  patterns  of  use,  can  be  developed  with  a  knowledge  of 
these  facts  on  hand,  A  second  objective  was  to  develop  a 
methodology  which  could  be  used  in  the  future  by  the  Naval  Training 
Device  Center  to  assess  the  acceptability  of  training  devices. 

Factors  found  to  Influence  acceptance  are  subsumed  under  six  main 
categories.  These  are:  (7 .  simulation  factors,  (2)  specific 
trainer  features,  (3)  characteristics  of  users,  (4)  characteristics 
of  instruction,  (5)  patterns  of  use,  and  (6)  the  manner  in  v.hich 
devices  are  introduced  to  the  user. 

Although  no  trainer  is  totally  "acceptable"  or  "unacceptable," 
improvement  in  the  above  listed  categories  will  result  in  higher 
acceptance  of  trainers  with  the  consequence  of  'improved  training. 

Specific  recommendations  to  increase  acceptance  and  improve 
methods  of  utilizing  complex  trainers  include:  (l)  greater  in¬ 
vestments  in  training  software,  (2)  providing  better  technical 
documentation,  (3)  providing  for  higher  levels  of  fidelity  in 
the  "feel"  of  the  simulated  vehicle,  (4)  developing  uniform  and 
high  standards  for  instructors,  and  (5)  developing  a  "trainer 
advocate"  for  devices  representing  a  major  investment. 

The  acceptance  profiling  technique  developed  for  this  study 
should  be  applied  periodically  to  all  complex  trainers  so  that 
a  body  of  knowledge  is  developed  that  will  be  useful  for  design 
decisions. 


Research  Psychologist 
Naval  Training  Device  Center 
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SECTION  I 
INTRODUCTION 


The  critical  role  of  training  devices  in  developing  and 
maintaining  the  skills  of  Navy  personnel  is  accepted  by  vir¬ 
tually  everyone.  Recognition  of  the  need  for  a  training 
device,  however,  does  not  automatically  insure  a  high  level 
of  acceptance  of  the  device  by  its  users.  Many  factors  op¬ 
erate,  some  perhaps  obvious  and  some  quite  subtle,  to  deter¬ 
mine  the  level  of  acceptance  and  method  of  use  of  training 
devices  by  military  personnel.  In  extreme  cases,  strong  pat¬ 
terns  of  rejection  can  develop  that  can  defeat  the  very  best 
of  intentions  and  large-scale  investments  in  training  systems. 

The  problem  is  not  peculiar  to  training  devices.  The 
Navy  has  become  increasingly  concerned  about  the  potential 
degradation  of  systems  performance  associated  with  non- 
acceptance  of  various  kinds  of  operational  equipment  by  the 
personnel  who  must  maintain  and  use  it.  Several  studies 
have  shown  that  in  the  hands  of  the  users,  new  equipment 
often  fails  to  achieve  its  full  potential  performance  capa¬ 
bility.  It  is  obviously  in  the  interest  of  all  to  identify 
the  factors  that  operate  in  the  acceptance  or  rejection  of 
Navy  hardware  systems  and  to  try  to  develop  techniques  whereby 
acceptance  can  be  maximized. 

In  the  economic  world,  the  process  of  acceptance  has  been 
a  major  concern  of  every  manufacturer  of  consumer  goods--how 
does  one  gain  acceptance  on  the  part  of  the  public  to  which 
he  wishes  to  sell  his  product?  In  the  marketplace,  the  conse¬ 
quences  of  consumer  resistance  are  immediate  and  economically 
severe.  In  the  military,  where  there  usually  is  no  competi¬ 
tion  (i.e.,  the  user  has  no  choice),  the  consequences  are  more 
subtle  but  very  likely  just  as  serious. 

It  was  the  objective  of  the  study  reported  here  to  (1) 
identify  the  factors  leading  to  the  acceptance  (or  rejection) 
of  training  devices  by  their  users;  and  (2)  to  determine  the 
extent  to  which  these  acceptance  factors  were  present  in 
"effective"  and  "ineffective"  training  devices. 

SOME  BACKGROUND  CONCERNING  THE  PROCESS  OF  ACCEPTANCE 

The  processes  of  acceptance  and  rejection  have  long  been 
studied  from  the  academic  viewpoint  by  the  social  scientists 
concerned  with  attitude  formation  and  change.  The  appropri¬ 
ate  literature  was  carefully  reviewed  in  an  effort  to  develop 
hypotheses  that  might  be  pertinent  to  the  acceptance  of  train¬ 
ing  devices.  Within  the  literature  there  was  also  a  small 
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number  of  studies  directly  concerned  v/ith  the  acceptance  of 
hardware  developments  by  Navy  personnel.  In  one  of  the 
earliest  such  studies,  Simpson  and  Parker  (18)  performed  a 
field  study  of  the  acceptance  of  the  ASPECT  sonar  system  by 
fleet  personnel.  The  investigation  was  concerned  both  with 
the  operaticnal  use  of  this  equipment  and  the  degree  to  which 
it  had  been  maintained  in  good  operational  order.  It  was' 
found  that  the  equipment  on  board  over  half  of  the  ships 
visited  was  either  incapable  of  producing  target  signals  at 
all,  produced  signals  of  very  poor  quality,  or  was  so  severely 
miscalibrated  that  proper  operation  was  impossible.  Simpson 
and  Parker  suggested  that  the  principal  problem  lay  in  the 
fact  that  a  lack  of  understanding  of  the  purpose  and  operation 
of  the  device  had  engendered  biases  against  it.  These  biases 
led  to  insufficient  maintenance,  thus  reducing  potential 
operational  effectiveness,  which  in  turn  further  strengthened 
the  biases,  and  so  on,  in  a  never-ending  cycle. 

A  number  of  investigators  have  stressed  the  importance 
of  firsthand  personal  experience,  in  contrast  to  the  simple 
presentation  of  factual  information,  in  promoting  the  ac¬ 
ceptance  of  new  equipment.  Berger  et  at.  (1)  suggested  that 
the  importance  of  personal  trial  was  the  result  of  the  oppor¬ 
tunity  to  overcome  negative  attitudes  based  on  unfamiliarity, 
to  compare  one’s  own  performance  using  the  new  equipment  with 
that  using  older  equipment,  to  build  confidence  in  how  to 
use  the  equipment,  and  finally  to  provide  an  opportunity  to 
communicate  attitudes  and  ideas  about  the  equipment  to  one’s 
peers . 

An  important  incidental  finding  of  Berger's  study  was 
that  attitudes  toward  new  equipment  may  change  over  time 
without  any  additional  firsthand  experience,  as  a  result  of 
informal  communication.  A  period  of  initial  resistance  to 
innovations  may  well  be  overcome, 'in  time,  provided  that 
the  proper  provisions  for  trial  have  been  made.  If  not, 
negative  attitudes  become  stabilized  and  possibly  increase 
in  strength  with  time. 

There  is  little  objective  documentation  of  user  accep¬ 
tance  problems  with  respect  to  training  devices.  However, 
in  an  earlier  NTDC  study  (20),  a  committee  of  engineers, 
training  specialists,  and  maintenance  personnel  investigated 
the  use  of  two  operational  flight  trainers.  The  trainers 
were  studied  from  the  viewpoint  of  how  effectively  they  were 
utilized  by  both  training  commands  and  the  fleet.  These  in¬ 
vestigators  identified  significant  differences  between  design 
capability  and  the  manner  in  which  the  trainers  were  actually 
used.  They  found  that  neither  of  the  operational  flight 
trainers  was  in  a  condition  that,  permitted  use  of  all  of  its 
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capabilities,  either  in  a  single  problem,  or  in  a  series  of 
problems.  It  was  found  that  despite  their  other  capabilities, 
the  OFT's  were  used  almost  exclusively  for  cockpit  familiar¬ 
ization  training  in  normal  and  emergency  procedures.  They 
reported  that  squadron  training  officers  for  the  most  part 
had  little  confidence  in  the  trainers,  and  did  not  as  a  rule 
make  OFT  instruction  a  part  of  the  training  syllabus.  It 
appeared  that  some  pilots  had  a  "generalized  dislike"  for 
simulators  as  a  result  of  early  negative  experiences.  It  was 
concluded  that  the  problem  of  OFT  utilization  night  be  prin¬ 
cipally  one  of  educating  the  users. 

In  many  respects,  the  problem  of  optimizing  the  utiliza¬ 
tion  of  a  training  device  is  similar  to  that  of  maximizing 
the  demand  for  a  consumer  product.  The  ultimate  user  of  a 
trainer  must  be  convinced  of  its  worth,  and  educated  con¬ 
cerning  its  capabilities  and  limitations.  Apathetic  accep¬ 
tance  may  lead  to  poor  methods  of  utilization  which,  in 
turn,  may  eventually  result  in  actual  rejection  of  the  train¬ 
ing  device.  The  latter  can  also  result  from  overselling  or 
misinformation  concerning  a  trainer's  capabilities. 

In  a  recent  study  of  the  acceptance  of  Navy  equipment, 
Mecherikoff  and  Mackie  (11)  showed  that  the  effects  of  de¬ 
sign  deficiencies  can  become  accentuated  through  personality 
dynamics.  One  of  the  most  important  dynamic  reactions  that 
clearly  led  to  rejection  was  the  "not  invented  here"  phenom¬ 
enon.  This  refers  to  the  formation  of  negative  attitudes 
that  can  result  from  the  absence  of  personal  involvement  in 
a  technical  innovation  in  Gne's  own  area  of  expertise.  Any 
deficiency  in  the  design  of  the  innovation  may  be  magnified 
in  importance  by  the  "not  invented  here"  reaction. 

Another  factor  identified  in  this  study  as  important  to 
the  acceptance  process  has  been  clearly  emphasized  in  the 
extensive  literature  on  attitude  formation  and  change.  Atti¬ 
tudes  toward  new  equipment  or  other  innovations  are  critically 
determined  by  the  "change  advocate"--some  individual  who 
functions,  formally  or  informally,  in  the  role  of  initial  com¬ 
municator  concerning  the  advantages  and  capabilities  of  the 
device  to  the  potential  users.  The  qualifications  and  other 
characteristics  of  the  change  advocate,  real  or  perceived, 
are  critically  important  to  the  development  of  acceptance, 
particularly  if  the  innovation  is  in  an  area  where  the  poten¬ 
tial  users  consider  themselves  experts.  Such  factors  as  the 
change  advocate's  credibility,  prestige,  perceived  motivation, 
relationship  to  the  users,  apparent  impartiality,  intent  to 
influence,  methods  of  handling  criticism,  etc.,  are  factors 
that  significantly  affect  his  success  in  promoting  the  accep¬ 
tance  of  new  equipment. 
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While  the  importance  of  the  change  advocate  has  been 
known  to  social  scientists  for  many  years,  it  is  apparent 
that  there  has  been  no  systematic  attempt  to  employ  well 
qualified  advocates  for  the  introduction  of  new  equipment 
into  the  Navy,  whether  that  equipment  be  operational  or 
training  equipment.  In  fact,  in  many  cases,  the  role  of 
change  advocate  is  never  filled  at  all,  with  the  not  unpre¬ 
dictable  consequence  of  disuse  or  misuse  of  the  equipment. 

In  some  cases,,  the  role  of  change  advocate  falls  by  default 
on  petty  officers  whose  primary  responsibility  toward  the 
training  device  is  its  operation  and  maintenance.  In  a  few 
instances,  the  role  of  change  advocate  has  been  unofficially 
assumed  by  some  highly  motivated  individual  who  recognizes 
the  need  to  solve  a  training  problem  and  is  willing  to  do 
something  about  it,  however  sophisticated  his  approach  may 
be.  But  in  many  cases,  there  is  no  qualified  advocate  at 
all,  official  or  otherwise. 

When  a  training  device  is  designed,  the  designers  make 
judgments  and  decisions  based  on  their  understanding  of 
important  parameters  of  the  total  situation.  These  hopefully 
include : 

1.  The  deficiencies  of  present  devices  (procedures 
or  systems) . 

2.  How  learning  occurs. 

3.  How  the  device  will  fit  into  an  existing 
curriculum  and  what  changes  in  curriculum 
may  be  required. 

4.  The  range  of  usefulness  of  the  device  (specifi¬ 
cally  what  it  is  intended  to  accomplish  ar.d 
what  it  is  not  intended  to  accomplish). 

It  seems  likely  that  the  designers  themselves  may  not 
be  fully  aware  of  the  extent  to  which  they  are  making  such 
decisions;  certainly,  the  assumption  is  doubtful  that  the 
eventual  users  understand  all  of  these  factors  in  the  same 
way  that  the  designer  does,  or  that  the  1  ardware  itself  will 
somehow  convey  its  purposes. 

McClelland  v'i0)  in  his  extensive  analysis  cf  the  pro¬ 
cesses  of  effecting  change,  has  identified  a  number  of 
fallacious  propositions  that  clearly  appear  to  be  pertinent 
to  the  introduction  of  training  devices  to  military  personnel: 

I.  A  good  product  will  succeed  on  its  own  merits. 

(Don't  you  believe  it.) 
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2.  The  introduction  of  a  new  device  is  a  finuL 
act.  No  further  attention  is  required.  (In 
the  absence  of  a  formula  for  maintenance  and 
feedback,  many  training  innovations  are  gath¬ 
ering  dust  and  teachers  and  managers  have 
reverted  to  former  practices.) 

3.  There  is  an  orderly  -process  from  research  to 
development  to  use.  (The  fallacy  of  this 
proposition  has  been  emphasized  by  numerous 
investigators . ) 

Quesada  (15)  has  identified  five  criteria  that  appear  to 
play  significant  roles  in  the  acceptance  of  innovations. 

These  have  been  paraphrased  with  reference  to  training  devices. 

1.  Relative  Advantage.  The  degree  to  which  the 
training  device  is  perceived  as  being  superior 
to  the  one  it  succeeds. 

2.  Compatibility.  The  degree  to  which  the  device 
is  perceived  as  consistent  with  operational 
requirements,  equipment,  and  past  experience. 

3.  Complexity .  The  degree  to  which  the  device  is 
relatively  difficult  or  easy  to  understand  and 
use . 

4.  Visibility .  The  degree  to  which  the  results 
of  using  the  device  may  be  transmitted  to 
others  in  a  way  that  is  easily  seen  or  demon¬ 
strated  . 

5.  Divisibility .  The  degree  to  which  the  device 
may  be  tried  on  a  limited  basis  to  gain  first¬ 
hand  personal  experience. 

It  seems  likely  that  all  or  "hese  criteria  apply,  in  one 
way  or  another,  in  determining  t  .e  user's  acceptance  of  Navy 
equipment  and  training  devices. 

Quesada  also  distinguishes  between  three  types  of  de¬ 
cisions  in  the  acceptance  process.  An  "authority  decision" 
results  when  those  high  in  the  power  structure  require  adop¬ 
tion  of  an  innovation.  A  "contingent  decision"  permits  the 
individual  to  adopt  or  reject  the  innovation  but  only  after 
an  enabling  decision  is  made  by  the  organization.  Finally, 
a  "collective  decision"  occurs  when  individuals  in  a  particu¬ 
lar  group  participate  in  the  verdict  concerning  a  given 
innovation.  Decisions  of  all  these  types  may  be  made  in 
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connection  with  an  individual's  use  of  a  training  device. 

They  need  not  be  mutually  exclusive.  An  authority  decision 
may  bs  made  that  requires  p-.-yj « lie  use  of  the  device,  but 
this  in  no  way  prevents  independent  individual  or  group  de¬ 
cisions  concerning  the  value  of  ming  the  device.  Clearly, 
true  acceptance  must  be  a  functor,  of  the  "collective  de¬ 
cision"  for  it  is  the  decision  at  the  user  level  that  even¬ 
tually  determines  whether  the  device  ‘.a  used  effectively  or 
not . 

Chin  (3)  has  a  similar  trichotomy  of  what 
strategies  for  change.  The  "power  approach"  ix:v? »  k  rn- 
pliance  and  submission  with  a  limiting  of  alternative:-  ,  *  d 
shaping  of  consequences  by  those  in  authoritative  . 

Chin  points  out  that  this  may  be  counterproductive  1 hv 
desired  acceptance  pattern.  A  second  strategy  is 
"normative-reeducative . "  This  approach  depends  on  the  .'0 
of  values  in  a  group  with  which  the  innovation  is  identified. 
An  example  of  this  approach  is  "sensitivity  training."  The 
third  is  the  "empirical-rational"  approach  which  depends 
heavily  on  demonstration  of  the  relative  merits  of  the  in¬ 
novation  . 

In  his  studies  of  factors  influencing  the  acceptance  of 
planned  changes  in  other  cultures,  Niehoff  (13)  has  studied 
the  transfer  of  innovations  to  new  cultural  groups.  A  number 
of  conditions  were  identified  that  helped  expedite  this 
process:  (1)  the  innovation  shot  Id  be  compatible  with  cul¬ 

tural  patterns  of  the  recipient  group  (the  amount  of  new 
behavior  required,  and  old  behavior  given  up  should  be  mini¬ 
mal)  ;  (2)  the  innovation  should  meet  existing  or  felt  needs 
of  the  recipient,  probably  those  they  have  tried  to  solve 
through  their  own  efforts;  (3)  the  innovation  should  provide 
practical  (economic)  benefits;  (4)  the  strategy  of  introduc¬ 
tion  should  involve  adapting  to  and  working  through  the  local 
cultural  patterns,  particularly  the  pattern  of  local  leader¬ 
ship;  (5)  channels  of  communication  should  be  established 
through  a  change  agent  who  provides  for  an  efficient  two-way 
flow  of  information  (feedback  from  the  recipients  to  the 
change  agent  is  particularly  vital);  (6)  the  recipients 
should  be  involved  in  the  introduction  process  through  full 
participation;  (7)  the  change  agent  should  be  flexible  in  his 
strategies;  (8)  the  change  agent  should  establish  patterns  of 
maintenance  (of  the  innovation)  among  the  recipients  so  that 
the  innovation  can  be  continued  when  his  influence  is  with¬ 
drawn. 

When  it  comes  to  innovations  in  training  systems,  it 
seems  likely  that  the  attitudes  of  the  instructor  (as  well 
as  his  competence)  can  be  critically  important  to  user 
acceptance.  The  literature  cn  teacher  attitudes  toward 
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various  approaches  to  instruction  and  training  media  is 
voluminous.  One  recent  study,  which  is  perhaps  typical,  was 
reported  by  Finch,  et  at.,  (6)  who  studied  the  attitudes  of 
100  instructors  in  vocational-tc^..nical  schools  to  determine 
interrelationships  among  teacher  attitudes  towai  ’«  resources, 
resource  utilization  and  availability.  The  results  indicated 
that  "the  teacher  group  generally  had  a  more  favorable  atti¬ 
tude  toward  ’traditional*  instructional  resources  as  opposed 
to  ’progressive'  materials.  The  traditional  materials  were 
used  more  often  and  were  more  readily  available.  Relation¬ 
ships  between  attitude  and  use  and  between  availability  and 
use  were  generally  positive  and  significant."  A  factor 
analysis  of  the  attitude  variables  revealed  that  "teachers 
may  view  instructional  resources  in  accordance  with  their 
personal  involvement  with  resource  preparation,  selection, 
presentation,  and  application.  The  results  indicate  that 
pre-service  and  in-service  experience  should  be  provided  to 
acquaint  teachers  with  new  resc  'ces." 


On  a  related  point,  Toye  (19)  makes  the  apparently  not 
so  obvious  observation  that  a  training  program  has  to  be 
capable  of  working  not  only  when  it  is  installed  but  nf  con¬ 
tinuing  to  work  despite  organizational  changes  and  accidents 
that  beset  any  operation.  "The  literature  on  training  and 
training  methods  rarely  mentions  how  a  training  scheme  works 
after  its  initial  validation."  As  will  be  seen,  this  is  a 
significant  observation  for  the  continued  successful  use  of 
Navy  training  devices. 
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SECTION  II 

SOME  SPECIFIC  HYPOTHESES  CONCERNING  ACCEPTANCE 


In  their  study  of  attitudinal  factors  influencing  the 
acceptance  of  new  equipment  in  the  Navy,  Mecherikoff  and 
Mackie  ( op .  ait.)  attempted  to  summarize  all  of  the  in¬ 
fluencing  factors  that  appear  to  operate  in  the  pattern  of 
acceptance  or  rejection.  These  factors  relate  in  part  to  the 
hardware,  in  part  to  the  processes  of  introducing  and  promot¬ 
ing  the  device  (advocacy) .  Not  all  of  the  factors  were 
considered  equally  important  or  even  applicable  in  every  case. 
However,  the  list  appears  fairly  exhaustive  of  the  factors 
that  should  be  considered  in  the  course  of  introducing  any 
new  equipment.  In  the  reproduction  of  this  list,  the  state¬ 
ments  have  been  paraphrased  as  they  might  apply  specifically 
to  training  devices. 

HARDWARE  OR  SYSTEM  FACTORS 

Conceptual  » 

1.  Agreement  on  definition  of  the  need  or  require¬ 
ment  for  a  particular  training  device. 

2.  Various  engineering  approaches  to  device  design. 

Physical  Factors 


1. 

Equipment  reliability  and 

maintainabi lity . 

2. 

Problems  of  mismatch  with 
vironment . 

the  operating  en- 

5 . 

Problems  of  mismatch  with 
user  personnel. 

the  capabilities  of 

4. 

Possibilities  of  mismatch 
of  the  training  system. 

with  other  elements 

Psychological  Factors 

1.  Reaction  to  the  appearance  of  the  training  device. 

2.  Perception  of  its  ’’fit"  into  the  training  en¬ 
vironment  . 

3.  Reactions  to  delays  in  delivery  (the  device 
itself,  components,  spare  parts,  software). 
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4.  Opinions  formed  on  the  basis  of  hearsay  and 
rumor . 

5.  Opinions  formed  on  the  basis  of  limited  ex¬ 
perience  with  the  device. 

Support 

1.  Documentation  of  the  purposes  and  functions 
of  the  device. 

2.  Documentation  on  how  the  device  should  be 
operated . 

3.  Documentation  of  technical  specifications  and 
maintenance  data. 

ADVOCACY  FACTORS 

Consideration  of: 

1.  What  different  kinds  of  persons  or  groups  will 
use  the  device  and  what  are  their  relevant 
characteristics . 

2.  What  communication  channels  were  provided  for 
user  inputs  during  the  design  phases. 

3.  What  means  were  provided  to  detect  and  resolve 
differences  in  approach  or  philosophy  concerning 
the  design  of  the  device. 

4.  What  users  likely  will  want  to  know  about  the 
device . 

a.  Overall  purpose. 

b.  Direct  and  indirect  benefits  to  themselves. 

c.  Benefits  to  the  Navy. 

d.  Data  oi*  reliability. 

e.  Real  or  apparent  drawbacks  compared  with 
earlier  trainers . 

f.  Real  or  apparent  advantages  compared  with 
earlier  trainers. 

g.  Adjustments  that  must  be  made  in  the  user's 
behavior  patterns. 

h.  How  these  adjustments  will  be  achieved  (formal 
retraining,  assumption  of  personal  responsi¬ 
bility,  etc.). 
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i.  New  responsibilities  that  are  entailed. 

j.  Present  responsibilities  that  are  to  be 
reassigned  to  others. 

k.  Who  these  other  persons  are  and  what 
preparation  will  be  given  for  carrying 
them  out. 

l.  How  those  in  the  chain  of  ommand  will 

be  made  aware  that  these  responsibilities 
have  been  met. 

5.  Who  is  to  be  in  charge  of  promoting  usage  of  the 
trainer  (trainer  advocate). 

6.  What  are  the  credentials  and  authority  of  the 
trainer  advocate. 

7.  What  threats  the  new  device  might  present. 

a.  Physical. 

b.  Threat  to  current  confidence  with  respect 
to  operating  skills. 

c.  Threat  to  other  aspects  of  the  user's  self- 
image,  prestige,  etc. 

8.  What  the  trainer  advocate  expects  of  the  poten¬ 
tial  users  and  what  the  users  expect  of  him. 

Are  tl  >  two  sets  of  expectations  explicit  and 
agreed  upon  by  both  parties? 

9.  What  provisions  there  are  for  direct  experience 
with  the  new  device. 

10.  Whether  the  introduction  communications  (both 
oral  and  written)  are  designed  for  maximum 
appropriateness  considering  the  characteristics 
of  the  users. 

A  simple  listing  of  factors,  however,  is  not  enough  to 
describe  the  process  of  acceptance  or  rejection.  Clearly, 
this  process  is  dynamic  and  many  of  the  factors  operate  in 
an  interactive  way.  An  attempt  has  been  made  to  summarize 
the  factors  hypothesized  to  influence  training  device  ac¬ 
ceptance,  and  their  interrelationships,  in  Figure  1.  It 
seems  likely  that  a  device  will  not  be  accepted  or  rejected 
in  all  details,  but  rather  will  have  certain  outstanding 
characteristics  or  deficiencies,  whose  combined  influences 
place  the  device  on  some  position  on  an  acceptance-rejection 
scale.  This  consideration  is  reflected  in  the  figure. 
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It  was  our  objective  to  try  to  identify  the  influence 
of  as  many  of  the  factors  shown  in  Figure  1  as  possible. 

We  assumed  from  the  outset  that  some  of  the  factors  in¬ 
fluencing  acceptance  would  be  subject  to  the  control  of 
design  engineers  and  others  would  not. 

influence'  OF  THE  TRAINING  SITUATION 

Perhaps  the  most  important  factors  influencing  training 
device  acceptance  insofar  as  the  training  situation  is  con¬ 
cerned  are  (1)  the  opportunity  for  training  in  the  opera¬ 
tional  environment  (or  lack  of  it),  and  (2)  certain  char¬ 
acteristics  of  the  equipment  or  the  environment  that  may  make 
the  regular  practice  of  some  skills  virtually  impossible  from 
either  a  safety  or  economic  viewpoint.  If  there  is  ample 
opportunity  for  practice  of  critical  skills  using  operational 
equipment,  the  trainer  is  not  likely  to  be  the  preferred 
training  vehicle.  However,  there  are  nearly  always  some  pro¬ 
cedures  that,  for  practical  reasons,  simply  cannot  be  sys¬ 
tematically  trained  during  routine  operations.  Examples 
include  drill  in  emergency  procedures  under  simulated 
catastrophic  conditions,  the  fi-ing  of  expensive  or  high 
yield  weapons,  and  the  manipulation  of  environmental  factors 
that  have  major  influences  on  operating  effectiveness,  such 
as  sonar  detection  ranges. 

Level  and  uniformity  of  skill  development  in  the  fleet 
is  an  obvious  influence  on  the  acceptance  and  use  of  train¬ 
ing  devices.  A  less  obvious  influence  is  the  amount  of  time 
actually  available  for  training  in  relation  to  operating 
demands  and  administrative  responsibilities  that  compete  for 
the  same  time.  This  is  a  major  consideration  in  evaluating 
statistical  data  that  may  reflect  trainer  acceptance  such  as 
percent  utilization  figures.  In  many  cases  fleet  personnel 
simply  are  not  able  to  utilize  a  trainer  to  the  extent  they 
would  like  because  of  administrative  duties.  Any  perceived 
deficiencies  in  the  trainer,  however,  may  serve  to  resolve 
the  conflict  between  administrative  demands  and  exercises  in 
the  trainer  in  favor  of  the  former. 

MILITARY  AND  DESIGN  SPECIFICATIONS 

It  was  beyond  the  scope  of  this  study  to  perform  a 
training  situation  analysis  for  all  of  the  trainers  whose 
acceptance  was  studied,  or  to  analyze  how  fleet  requirements 
and  the  inputs  of  educational  specialists  and  engineers  were 
translated,  first  into  military  characteristics,  and  then 
into  training  device  design  specifications.  We  felt  it 
necessary,  however,  to  examine  the  military  characteristics 
for  each  trainer  to  identify  its  general  design  objectives. 


NAVTRADEVCEN  70-C-0276-1 


Clearly,  the  question  of  trainer  acceptance  must  be  related 
to  what  it  was  intended  to  do  and  whether  or  not  the  users 
perceive  it  as  fulfilling  the  same  role  as  the  designers  did. 
It  seems  quite  possible  that  a  trainer  might  be  accepted  (or 
rejected)  for  a  particular  pattern  of  use  which  might  be 
qualitatively  different  from  the  purposes  stated  in  the 
military  characteristics. 

SIMULATION  FACTORS 

Prophet  (14)  has  pointed  out  that  a  training  device  has 
one  principal  purpose--to  provide  an  environment  in  which  one 
can  be  trained.  The  most  obvious  characteristics  of  * 
trainer  that  influence  its  acceptance,  then,  are  those  fea¬ 
tures  of  the  operating  environment  that  have  been  selected 
for  simulation  and  the  fidelity  of  such  simulation.  Prophet 
has  emphasized  that  simulators  do  not  train;  rather,  *' t  is 
the  training  program  that  trains.  The  critical  factor  is 
task  fidelity .  While  we  fundamentally  agree  with  this  view¬ 
point,  it  is  nonetheless  true  that  the  features  selected  for 
simulation  determine  important  boundaries  within  which  the 
training  must  be  accomplished.  In  fact  they  determine  the 
extent  to  which  task  fidelity  is  possible.  We  felt  it  was 
important  therefore  to  identify  all  major  areas  of  simula¬ 
tion  applicable  to  major  training  devices  and  to  determine 
user  impressions  of  the  adequacy  oi  that  simulation.  The 
factors  selected  for  study  were  as  follows: 

SM-1.  Simulation  of  the  Internal  Operating  Environment. 
This  factor  was  concerned  with  the  adequacy  of  simula- 
tion  of  such  characteristics  of  the  internal  operating 
environment  as  the  type  and  location  of  equipment, 
arrangement  of  displays  and  controls,  illumination, 
noise,  motion,  temperature,  vibration,  and  other  in¬ 
ternal  features  that  might  be  considered  critical  for 
training. 

SM-2.  Simulation  of  the  External  Environment.  This 
included  consideration  of  such  features  as  the  visual 
scene,  auditory  stimuli  or  noise  from  the  external 
environment,  and  ambient  temperature,  pressure, 
atmospheric  or  water  conditions  that  might  affect 
equipment  or  personnel  performance. 

SM-3.  Simulation  of  Vehicle/Equipment  Performance. 

This  factor  was  concerned  with  how  well  the  performance 
characteristics  of  the  vehicle  or  equipment  were 
simulated.  In  the  case  of  vehicles,  consideration  was 
to  be  given  to  maneuverability,  response  time,  and 
range  of  operation;  in  the  case  of  equipment,  considera¬ 
tion  centered  on  accuracy  of  operation,  limits  of 
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operational  effectiveness,  typical  operating  difficulties, 
etc . 

SM-4.  Simulation  of  Controls.  Control  effec's  were  to 
be  considered  from  the  standpoint  of  the  "feel"  of  the 
simulated  vehicle  and  the  effe  :ts  of  control  manipula¬ 
tions  on  how  the  system  operate '  or  how  various  in¬ 
formation  was  developed  and  displayed. 

SM-5.  Simulation  of  Communication  Procedures.  This  was 
concerned  with  the  various  com&tnication  links  and  pro¬ 
cedures  in  the  trainer  with  resp  ;Ct  to  communications 
both  within  and  between  operational  units. 

SM-6.  Simulation  of  Communication  Problems.  Considera¬ 
tion  was  to  be  given  to  snch  problems  as  environmental 
noise  that  might  interfere  with  communications,  faulty 
equipment  and  various  other  sources  of  interference 
with  the  communicated  information. 

SM-7.  Simulation  of  Information  Displays.  This  was 
concerned  with  the  fidelity  of  information  as  displayed 
on  scopes,  plots,  dials,  and  status  boards.  Both  video 
and  audio  displays  were  to  be  considered. 

SM-8.  Simulation  of  Sensor  Performance.  Consideration 
was  to  be  given  to  such  factors  as  the  physical  repre¬ 
sentation  of  objects  or  conditions  in  the  environment 
as  sensed  through  the  equipment;  the  number  and  variety 
of  objects  simulated;  how  detectable  the  objects  were 
as  a  function  of  environmental  conditions,  etc.  If 
applicable,  consideration  was  to  be  given  to  the  dif¬ 
ferent  classes  of  targets  that  were  simulated. 

SM-9.  Target  Evasion.  For  trainers  that  included 
target  simulation,  consideration  was  given  to  how 
realistically  the  target  employed  evasive  tactics, 
agressive  tactics,  and  countermeasures. 

SM-10.  Simulation  of  Weapon  Firing  and  Control.  For 
trainers'  that  involved  the  firing  or  control  of  weapons, 
consideration  was  to  be  given  to  the  simulation  of  tar¬ 
get  tracking,  localization,  solution  to  the  fire-control 
problem,  choice  of  weapons,  weapon  launching  procedures, 
and  real’  .sm  of  the  results  of  the  attack  in  terms  of 
typical  weapon  effectiveness. 

TRAINING  EFFECTIVENESS  AND  SITUATIONAL  FACTORS 

Although  the  simulated  operational  environment  provided 
by  a  training  device  is  undoubtedly  a  major  consideration  in 
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training  device  acceptance,  it  is  by  no  means  the  only  char¬ 
acteristic  of  major  importance.  In  discussing  the  importance 
of  training  requirements  information  in  the  design  and  use  of 
aviation  training  devices.  Prophet  (pp.  ait.)  contended  that 
the  "usual"  approach  is  to  provide  the  engineer  with  cockpit 
and  performance  specifications  and  let  him  build  the  device. 
Little  mention  may  be  made  of  what  is  to  be  learned,  the 
psychological  nature  of  the  tasks  involved.  For  Prophet,  the 
most  important  characteristic  of  an  effective  trainer  is  "task 
fidelity."  '.he  achievement  of  task  fidelity  is  made  more 
likely  by  proper  simulation  but  other  Factors  must  be  taken 
into  account.  Further,  the  fidelity  c  simulation  desired  in 
a  trainer  is  in  part  a  function  of  situational  factors  in  the 
operational  environment. 

A  number  of  situational  and  training  requirements  factors 
were  hypothesized  to  be  important  to  training  device  accep¬ 
tance.  These  were  as  follows: 

TSF-1.  Problem  Setup.  Within  the  purpose  for  which  the 
trainer  was  designed,  how  readily  can  a  variety  of 
operational  problems  be  set  up  on  the  trainer?  Con¬ 
sideration  was  to  be  given  to  representativeness  and 
scope  of  problems  in  relation  to  operational  require¬ 
ments,  and  the  relative  ease  or  difficulty  of  setting 
up  the  problem  for  execution. 

TSF-2.  Software .  Consideration  was  to  be  given  to  the 
thoroughness,  effectiveness,  and  sophistication  of  the 
utilization  guides  and  other  training  materials  asso¬ 
ciated  with  the  trainer.  The  question  was  asked, 

"How  effectively  could  the  trainer  be  used  if  only  the 
information  contained  in  these  guides  were  used  to 
program  and  operate  it?"  (Technical  and  maintenance 
manuals  were  not  to  be  included  in  this  factor.) 

TSF-3.  Completeness  of  Performance  Evaluation.  Con¬ 
sideration  was  to  be-  given  to  the  trainer's  provisions 
for  providing  information  to  individual  students  con¬ 
cerning  all  important  elements  of  their  performance. 

TSF-4.  Immediacy  of  Performance  Evaluation.  The 
trainer's  provisions  for  immediate  feedback  were  to 
be  considered  in  contrast  to  situations  where  there 
is  feedback  only  after  extensive  delay. 

TSF-5.  Reliability  and  Maintainability.  Consideration 
was  to  be  given  to  the  amount  of  down  time,  and  the 
level  of  effort  required  to  keep  the  trainer  operational. 
How  readily  can  it  be  kept  operational  by  the  average 
training  device  technician? 
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TSF-6.  Level  of  Training.  Consideration  was  to  be  given 
to  whether  the  trainer  was  suitable  mainly  for  basic 
training  and  familiarization  or  whether  it  could  be  used 
effectively  for  advanced  or  refresher  training  as  well. 

TSF-7.  Training  Opportunities.  Consideration  was  given 
to  the  skills  developed  by  the  trainer  in  relation  to 
the  opportunity  for  acquiring  and  maintaining  those  same 
skills  in  the  operational  environment. 

TSF-8,  Comprehensiveness  of  Training.  Trainer  capa- 
bi lities~were  to  be  considered  in  relation  to  all  of 
the  skills  that  must  be  acquired  for  effective  perfor¬ 
mance  in  a  selected  area  of  Navy  operations.  Are  there 
any  critical  skills  that  receive  insufficient  emphasis? 

TSF-9.  v'alue  of  Time  in  the  Trainer..  In  relation  to 
all  the  activities  that  occur  in  the  trainer ,  what 
proportion  of  time  is  spent  in  actually  acquiring  or 
improving  essential  skills. 

TSF-10.  Use  of  the  Trainer  vs.  Operational  Equipment. 

One  of  the  recognized  advantages  of  trainers  is  that 
they  permit  training  on  activities  that  are  either  im¬ 
practical,  too  costly,  or  too  dangerous  to  perform  on 
actual  equipment.  The  disadvantage,  however,  is  that 
the  trainer  may  differ  in  some  significant  way  from 
actual  operational  gear.  The  respondent  was  to  consider 
how  much  advantage  there  is  to  using  the  trainer  as 
configured  vs.  using  actual  operational  equipment  for 
training  at  shore-based  schools. 

TSF-11.  Repeated  Use  of  the  Trainer.  When  a  trainer  is 
used  on  some  regular  basis,  at  least  some  of  the  skills 
may  already  be  well  trained.  Consideration  was  to  be 
given  to  how  useful  the  trainer  is  fGr  repeated  use  in 
maintaining  or  refreshing  the  essential  skills. 

TSF-12.  Comparison  with  Similar  Trainers.  From  an  over¬ 
all  training  viewpoint,  the  respondent  was  to  compare  the 
trainer  in  sophistication  and  effectiveness  with  other 
trainers  that  simulated  similar  Navy  systems. 

UNDERSTANDING  THE  PURPOSES,  CAPABILITIES.  AND  LIMITATIONS  OF 
THE  TRAINER 

The  literature  on  the  acceptance  of  innovations  clearly 
reflects  the  dual  roles  of  communications,  either  formal  or 
informal,  and  direct  personal  experience  wi-,h  the  innovation, 
in  developing  patterns  of  acceptance  or  rejection.  We  were 
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interested  in  three  basic  kinds  of  communications  in  relation 
to  training  devices  and  attempted,  insofar  as  possible,  to 
identify  their  existence  and  the  possible  roles  played  in 
generating  each  trainer's  reputation. 

1.  Formal  or  informal  statements  of  trainer  capa¬ 
bilities  . 

2.  Official  directives  concerning  use  of  the  trainer. 

3.  The  actions  of  individuals  who  were  openly  pro¬ 
moting  the  use  of  the  trainer  (trainer  advocates). 

It  was  assumed  that  direct  personal  use  of  a  trainer 
would  be  a  major  factor  governing  level  of  acceptance.  It 
is  insufficient,  however,  simply  to  determine  whether  the 
trainer  has  been  used  by  a  particular  individual,  or  how 
frequently  he  has  used  it,  unless  there  is  inquiry  also  into 
the  manner  of  use.  As  suggested  earlier,  it  is  quite  possible 
for  a  trainer  to  be  used  in  one  fashion,  for  example  as  a 
procedures  trainer,  when  the  military  characteristics  call 
for  an  operations  trainer  or  a  weapons  system  trainer.  Con¬ 
sequently,  an  effort  was  made  during  the  study  to  distinguish 
between  acceptance  of  a  trainer,  as  used,  versus  acceptance 
as  designed.  This  included  an  inquiry  into  the  qualifications 
of  instructor  personnel  who  operated  the  trainer  and  the  level 
of  satisfaction  felt  by  the  trainee  as  a  result  of  his  experi¬ 
ence  in  it. 

CHARACTERISTICS  OF  THE  USER 

The  literature  on  attitude  formation  suggests  that  many 
"psychological"  factors  are  involved  in  the  acceptance  of 
innovations  (Mackie  and  Mecherikoff,  op.  ait.).  Examples  in¬ 
clude  such  individual  characteristics  as  self-esteem,  level 
of  chronic  anxiety,  cognitive  rigidity,  need  for  authority 
approval,  status  in  group,  etc.  Some  of  these  factors  are 
extremely  subtle  and  adequate  techniques  for  their  measure¬ 
ment  probably  do  not  yet  exist.  However,  it  was  felt  that 
some  user  characteristics  could  be  measured  reasonably  objec¬ 
tively  and  an  attempt  was  made  to  do  so  in  the  following 
areas : 

1 .  Felt  level  of  personal  competence  in  the  technical 
area  represented  by  the  trainer.  Except  in  the 
case  of  neophytes,  each  user  of  a  training  device 
has  a  different  backg;ound  of  experience,  compared 
to  other  Navy  personnel,  that  possibly  affects  his 
attitude  toward  the  trainer. 
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Sufficiency  of  operational  training.  It  was  hy- 
polKesized  that  a  major  influence  determing  atti¬ 
tudes  toward  training  devices  was  the  extent  co 
which  the  user  felt  that  sufficient  training  was 
obtainable  in  the  typical  operational  environment. 
Some  Navy  personnel  seem  to  develop  biases  against 
trainers  because  of  the  assumption  that  the  neces¬ 
sary  operational  skills  can  only  be  obtained  in 
the  operational  environment. 

Cognitive  simplicity -complexity.  This  refers  to 
the' rel“tive  rigidity  or  flexibility  with  which  an 
Individ  .al  regards  a  particular  innovation.  Cog¬ 
nitive  simplicity  is  exhibited  by  the  tendency  to 
categorize  things  as  either  all  white  or  all  black, 
or  the  inability  to  consider  more  than  one  dimension 
of  an  argument  at  a  time.  In  the  context  of  train¬ 
ing  device  acceptance,  cognitive  simplicity  seemed 
to  be  reflected  in  the  viewpoint  of  some  that  unless 
the  trainer  were  perfect  in  all  respects,  its  use 
for  any  reason  was  a  waste  of  personal  time. 

Experience  with  other  trainers  of  this  type.  Ex¬ 
perience  with  other  ti'siners  results  in  a  set  of 
personal  expectations,  often  predominantly  positive 
or  negative,  that  was  thought  to  condition  acceptance 
of  a  new  trainer. 

Personal  involvement  with  the  design  or  original 
requirement  for  the  trainer .  The  negative  influence 
of  the  “^hot-inventedTere'1  phenomenon  in  developing 
attitudes  toward  innovations  has  been  mentioned  pre¬ 
viously.  Personal  participation  in  the  development 
of  an  innovation  and  the  opportunity  to  contribute 
to  the  solution  of  a  problem  in  one’s  own  area  of 
expertise  plays  a  highly  significant  role  in  accep¬ 
tance.  The  great  majority  of  Navy  personnel  who  use 
training  devices  have,  of  course,  no  opportunity  to 
contribute  to  the  design  of  that  device.  Neverthe¬ 
less,  tney  are  continually  passing  judgment  on  how 
well  someone  else  designed  the  device. 

ACCEPTANCE  DETERMINED  BY  CONGRUENCE  BETWEEN  PERSONAL  NEEDS 
AND  TRAINER  FEATURES 

At  the  final  stage  in  the  dynamic  process  depicted  in 
Figure  1,  it  was  hypothesized  that  the  level  of  acceptance 
for  any  particular  trainer  was  a  complex  function  of  the  felt 
needs  of  the  individual  (both  technical  and  psychological), 
and  the  perceived  value  and  deficiencies  of  the  trainer 
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resulting  from  various  types  of  communications  and  direct  per¬ 
sonal  experience.  The  word  perceived  is  emphasized  because 
we  did  not  assume  that  the  judged  characteristics,  value,  or 
deficiencies  of  the  trainer  would  necessarily  be  based  on 
objective  fact.  Further,  there  was  reason  to  believe  that 
most  users  lacked  detailed  knowledge  of  the  design  objectives 
and  military  characteristics  of  the  trainers. 


There  was,  of  course. 
However,  it  was  possible  t 
reflected  deficiencies  in 
to  the  felt  importance  of 
training.  The  methodology 
following  section. 


no  direct  measure  of  congruence, 
o  obtain  measures  that  apparently 
trainer  characteristics  in  relation 
those  characteristics  for  effective 
employed  is  described  in  the 
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SECTION  III 
STUDY  METHODOLOGY 


To  summarize  the  considerations  set  forth  in  the  previous 
section,  this  study  was  particularly  concerned  with  five  major 
sources  of  influence  on  the  process  of  trainer  acceptance: 

1.  The  training  situation,  particularly  the  level  and 
uniformity  of  skill  development  in  the  fleet;  the 
opportunities  for  training  certain  functions  in 

the  operational  environment;  various  characteristics 
of  the  equipment  and  environment  that  influence  the 
opportunity  to  exercise  particular  skills;  and  the 
training  time  available  in  contrast  to  operating  and 
administrative  demands. 

2.  Trainer  characteristics.  Two  broad  areas  were 

considered :  (a)  simulation  factors;  (b)  training 

effectiveness  and  situational  factors,  including 
convenience,  location,  availability,  and  reliability. 

3.  Understanding  of  the  trainer.  The  user's  comprehen¬ 
sion  of  the  trainer's  capabilities,  limitations,  and 
purpose;  knowledge  of  the  trainer  through  both  formal 
and  informal  communications  and  through  direct  per¬ 
sonal  use. 

4.  User  characteristics.  The  user's  felt  level  of  con¬ 
fidence  in  the  technical  area  represented  by  the 
trainer;  his  experi'nce  with  other  similar  trainers; 
his  personal  involvement  with  the  design  or  suc¬ 
cessful  use  of  the  trainer;  his  assessment  of  the 
training  need. 

5.  Congruence ♦  The  degree  of  conformity  (or  disparity) 
between  (a)  the  physical,  characteristics  of  the 
trainer;  (b)  the  user's  understanding  of  its  capa¬ 
bilities  and  limitations;  and  (c)  how  important,  he 
judged  these  various  characteristics  to  be  for 
effective  training. 

The  investigation  required  the  development  of  a  means  to 
obtain  scaled  judgments  of  all  trainer  and  user  character¬ 
istics  hypothesized  to  exert  major  influences  on  the  accep¬ 
tance  process.  It  was  desired  to  develop  definitive  profiler, 
of  different  trainers  that,  on  the  basis  of  various  opera¬ 
tional  criteria,  could  be  shown  to  be  differentially  "accepted" 
or  "rejected,"  "effective"  or  "ineffective." 
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To  accomplish  these  objectives,  the  following  procedures 
were  followed: 

1.  Specially  designed  quantitative  rating  scales  were 
developed  that  could  be  used  to  rate  training  de¬ 
vices  on  10  areas  of  simulation  and  12  training 
effectiveness  and  situational  factors. 

2.  A  structured  questionnaire  was  developed  to  provide 
detailed  information  that  might  amplify  the  ratings. 

3.  A  group  of  training  devices  was  identified  whose 
members  were  thought  to  reflect  significant  dif¬ 
ferences  in  level  of  acceptance  as  well  as  various 
kinds  of  operational  training. 

4.  Qualified  groups  of  users  were  identified,  con¬ 
tacted  and  interviewed  using  the  rating  scales  and 
structured  questionnaires.  To  insure  that  various 
viewpoints  were  adequately  represented,  the  user 
sample  was  carefully  selected  from  administrative, 
instructor,  student,  and  maintenance  personnel 
associated  with  each  trainer. 

5.  An  operational  definition  of  acceptance  (rejection) 
had  to  be  developed  that  reflected,  in  some  logical 
way,  all  available  objective  and  subjective  data. 

In  addition  to  opinion  data,  it  was  considered 
desirable  to  include  objective  indices  such  as  per¬ 
cent  utilization  and  the  percent  of  requested 
training  accomplished.* 

6.  A  technique  had  to  be  developed  whereby  some  of  the 
more  subtle  factors,  such  as  felt  level  of  com¬ 
petence,  and  the  perceived  importance  of  various 
characteristics  of  the  trainer  could  be  conveniently 
identified  and  quantified. 

SELECTION  OF  TRAINING  DEVICES  FOR  INCLUSION  IN  THE  STUDY 

The  selection  of  devices  for  inclusion  in  the  study  was 
a  critical  step  in  the  investigation.  On  the  one  hand,  the 
sample  size  had  to  be  large  enough  that  one  could  feel  con¬ 
fident  about  generalizing  the  results  to  the  population  of 
trainers.  Clearly,  this  objective  could  not  he  reached  if 
only  two  or  three  trainers  were  rather  intensively  studied. 


*This  was  defined  as  the  number  of  hours  used  divided  by  the 
number  of  hours  requested. 
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The  possibility  existed  that  there  might  be  different  con¬ 
siderations  with  respect  to  comparatively  simple  part-task 
trainers  from  those  pertaining  to  very  complex  weapons  sys¬ 
tems  trainers.  Another  issue  was  whether  trainers  designed 
for  air  operations  would  involve  the  same  kinds  of  acceptance 
criteria  as  trainers  oriented  toward  surface  ship  or  sub¬ 
marine  training. 

For  the  study  to  be  of  maximum  value,  it  was  decided 
first  that  the  sample  should  include  trainers  that  were  rela¬ 
tively  complex  and  represented  comparatively  large  investments 
either  in  terms  of  their  size  or  the  number  of  installations 
throughout  the  Naval  establishment.  Second,  it  was  considered 
necessary  to  include  trainers  that  represented  air,  surface, 
and  subsurface  operations.  Third,  a  decision  was  made  to 
include  trainers  which  were  known,  or  suspected,  on  the  basis 
of  a  priori  information,  to  represent  cases  of  both  high  and 
questionable  acceptance.  (In  one  case  there  was  evidence  of 
outright  rejection.)  Finally,  it  was  decided  that  a  few 
trainers  should  be  included  where  the  installations  at  two 
different  locations  represented  certain  hnown  design  differ¬ 
ences.  This  would  make  it  possible  to  determine  whether  the 
measurement  techniques  were  sensitive  to  these  differences. 

It  was  not  possible,  however,  to  establish  level  of  ac¬ 
ceptance  on  any  thorough  basis  prior  to  the  investigation. 

For  the  most  part,  the  only  a  priori  information  available 
to  the  investigators  were  the  general  impressions  o.f  educa¬ 
tional  specialists  and  engineers  at  NTDC  together  •«ith  some 
available  data  on  the  extent  of  utilization.  Th',  latter 
criterion  was  recognized  as  an  imperfect  index  of  acceptance 
because  of  the  many  practical  considerations  chat  influence 
trainer  utilization  that  have  little  to  do  with  acceptance. 

It  was  decided,  therefore,  that  a  posteriori  criteria  would 
have  to  be  developed  from  the  study  data  to  help  identify 
"effective”  and  "ineffective"  trainers.  In  addition  to  the 
criteria  already  mentioned,  these  would  include  the  method 
in  which  the  trainer  was  actually  being  used  (i.e.,  in  rela¬ 
tion  to  its  design  objectives)  and  the  various  expressions 
of  user  attitudes  toward  the  value  of  time  spent  in  the 
trainer. 

The  sample  of  \16  training  devices  selected  for  the  study 
is  listed  in  Table  1.  It  will  be  evident  that  the  trainers 
were  very  heterogeneous  in  nature,  involving  surface,  sub¬ 
surface,  and  airborne  operations,  and  including  devices  in¬ 
tended  for  part-task  training  of  individual  operators  up 
through  some  of  the  most  complex  coordinated  tactics  trainers 
ever  developed. 
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TABLE  1.  SAMPLE  OF  TRAINING  DEVICES 


Desi qnator 


pons 

terns 

Iner 


S-2E  aircrew 


S-2E  a i rcrew 


F-4B  a i rcrew 


E-2B  a i rcrew 


P-3  a i rcrew 


P-3  a i rcrew 


SS(N)  S/M  crew 


ocation 
of  Unit  Studied) 


North  Island 


uonset  Point 


Mi ramar 


North  Island 


Barber ' s  Point 


Moffett  Field 


Pearl  Harbor 


Trainers  2F64B 


2B2 


SH-3D  he  I i copter 


T-28  I nstrument 
trainer 


Carrier  approach 
land! n - 


Quonset  Point 


Pensaco I  a 


Pensaco I  a 


Tactl ca  1 

Team 

Trainers  I4A2 

ASW  team  trainer 

San  Diego 

1  4A6A 

Coordinated  ASW 
tact i cs 

San  Diego 

Sensor 

Operator 

Trainers 

I4B35 

JULI E/JEZ 
operat I ons 

Moffett  Field 

I4B40 

Radar/Mad 
operat ! ons 

Pax  River 

1  5EI6 

E 1 ectron i c 
countermeasures 

Moffett  Field 

Emergency 

Procedures 

Trainer 

2  IC5 

Submarine  casualty 
contro 1 

Pearl  Harbor 
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PROCEDURE 

The  basic  procedure  followed  during  the  study  was  for 
one  or  more  members  of  the  HFR  staff  to  visit  a  trainer 
installation,  usually  for  a  period  of  several  days.  During 
this  time,  project  personnel  became  acquainted  with  the 
trainer,  how  it  was  being  used,  and  the  instructor  and  main¬ 
tenance  personnel  assigned  to  it.  Arrangements  were  made 
to  administer  the  rating  scales  and  questionnaires  to  key 
administrative,  instructor,  and  maintenance  personnel  and  to 
a  representative  group  of  students. 

The  rating  scale  and  questionnaire  were  designed  so  that 
they  could  be  self-administered.  However,  it  was  felt  that 
the  resulting  data  would  be  much  more  reliable  if  each  in¬ 
dividual  participant  was  personally  instructed  by  a  project 
member  in  the  objectives  of  the  study,  and  the  mechanics  of 
completing  the  rating  scale  and  questionnaire.*  Wherever 
possible  this  was  done  with  groups  of  individuals  and  the 
participants  were  encouraged  to  complete  the  data  forms,  in¬ 
dependently,  asking  questions  of  the  project  representatives 
as  they  arose.  In  some  cases  it  was  necessary  to  permit  the 
respondent  to  complete  the  data  forms  at  their  convenience 
and  later  return  them  to  the  project  representative  at  the 
trainer  site.  On  the  average,  about  twenty-five  minutes  was 
required  to  complete  the  rating  scale  and  thirty-five  or 
forty  minutes  for  the  questionnaire. 

Upon  receiving  the  completed  materials,  the  project 
representative  immediately  reviewed  all  ratings  and  responses 
to  various  key  questions  (such  as  statements  about  defi¬ 
ciencies  in  the  trainer,  recommendations  for  changes  or 
improvements,  etc.)  f.or  the  purpose  of  eliciting  additional 
detailed  information  about  attitudes,  both  positive  and 
negative,  toward  the  trainer.  In  a  few  cases,  these  inter¬ 
views  were  short  and  perfunctory;  in  many  cases,  however, 
they  generated  extended  commentary  by  the  participants  in 
discussions  that  sometimes  lasted  an  hour  or  more  in  length. 
The  time  required  for  complete  data  collection  at  a  trainer 
site,  involving  on  the  average  about  twenty  participants, 
varied  from  as  little  as  two  or  three  days  to  as  long  as  a 
week  depending  on  the  availability  of  personnel.  For  ex¬ 
ample,  in  some  of  the  complex  team  trainers,  the  trainees 
could  not  be  interviewed  at  the  trainer  site  but  rather  had 
to  be  located  on  individual  ships  at  a  time  when  their  usually 
busy  work  schedules  permitted  an  opportunity  for  discussion. 


^Examples  of  the  Rating  Scale  and  Questionnaires  are  given 
in  Appendix  A. 
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A  description  of  the  326  participants  in  the  study  is 
contained  in  Table  2.  They  varied  greatly  in  rank  and  in 
personal  experience  in  the  area  of  operations  represented 
by  the  trainer.  They  were  classified  as  having  low,  medium, 
or  high  experience  in  accordance  with  the  distribution  of 
self-ratings  of  experience  shown  in  Table  2. 

Most  had  had  considerable  experience  with  the  trainer 
they  were  called  upon  to  rate.  Fifty-five  percent  of  the 
respondents  knew  the  trainer  in  the  role  of  student;  the 
remaining  45%  was  comprised  of  instructors,  maintenance 
personnel,  and  administrators.  The  number  of  people  shown 
in  Part  C  of  Table  2  exceeds  the  number  of  trainees  shown 
in  Part  A  because  some  administrative  and  instructor  person¬ 
nel  had  also  used  the  trainer  in  the  role  of  student. 

For  each  trainer  that  was  designed  for  team  training 
an  attempt  was  made  to  obtain  opinions  from  four  levels  of 
personnel : 

1.  Officers  with  command  responsibility. 

2.  Officers  having  responsibility  for  technical 
specialties  (for  example,  CIC  officers,  ASW 
officers,  TACCO’S,  engineering  officers,  etc.). 

3.  Senior  or  supervising  petty  officers. 

4.  Basic  operators  of  major  units  of  equipment. 

With  respect  to  non-students,  an  attempt  was  made  to 
obtain  the  opinions  of  the  following  types: 

1.  Administrative  personnel  (the  officer  in  charge 
of  the  training  facility,  the  operational  staff 
training  officer,  the  school  training  officer, 
etc. ) . 

2.  Instructor  personnel.  The  principal  instructors 
for  all  stations,  both  officer  and  enlisted. 

3.  Maintenance  personnel  (the  electronics  material 
officer  having  responsibility  for  the  device, 
as  well  as  the  leading  enlisted  maintenance 
personnel).  In  some  cases  these  were  Training 
Devicemen;  in  others,  they  were  ET's,  Stf's, 

OS's.  etc. 
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Since  it  was  hypothesized  that  acceptance  of  a  training 
device  might  be  a  function  of  how  expert  the  participants 
considered  themselves  to  be  in  the  technical  area  represented 
by  the  trainer,  all  participants  in  the  study  were  asked  to 
rate  themselves  on  a  0-to-iS  point  scale  with  respect  to  the 
following  statement: 

"Compared  with  other  Navy  personnel  who  work 
in  the  area  of  operations  or  type  of  equip¬ 
ment  represented  by  this  trainer,  how 
extensive  is  your  experience?  Consider  the 
number  of  closely  related  billets  you  have 
had  and  recency  of  experience." 

The  distribution  of  ratings  on  this  variable  are  shown 
in  Part  D  of  Table  2.  It  seems  evident  from  the  wide  dis¬ 
tribution  of  scores  that  the  participants  were  quite  candid 
in  reporting  their  experience.  To  interpret  this  result, 
the  following  definitions  of  anchor  points  which  appeared 
on.  the  scale  will  be  helpful. 

0  -  My  experience  in  this  area  of  operations  or  with 
this  equipment  is  nil. 

5  -  My  experience  is  somewhat  limited;  the  majority 
of  my  assignments  have'  been  in  other  areas  or 
with  other  equipments . 

10  -  My  experience  is  substantial;  although  I  have 

had  other  types  of  assignments,  I  have  had  more 
in  this  area  of  operations,  or  with  this  type 
of  equipment  than  any  other. 

15  -My  experience  is  extensive;  I  consider  this 

operational  area,  or  particular  type  of  equip¬ 
ment,  my  special  field. 

For  the  purpose  of  some  data  analyses,  the  participants 
were  divided  into  three  experience  groups  at  the  points  shown 
in  the  Table.  Those  classified  as  "low"  all  rated  themselves 
below  the  midpoint  of  the  scale,  the  majority  describing  them¬ 
selves  as  being  somewhat  limited  in  experience  with  this  type 
of  equipment,  A  large  middle  group,  who  rated  themselves  from 
just  above  the  midpoint  of  the  scale  to  about  the  three-quarter 
point,  were  classified  as  being  of  "medium"  experience.  Fin¬ 
ally,  a  third  group  that  rated  themselves  in  the  upper  one- 
fourth  of  the  scale  were  classified  as  "high"  in  experience. 
While  this  method  of  classification  was  somev/hat  arbitrary, 
it  was  felt  to  be  adequate  for  testing  the  general  hypothesis 
that  level  of  experience  is  an  influencing  variable  insofar 
as  trainer  acceptance  is  concerned. 
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SECTION  IV 

DEVELOPMENT  OF  AN  OPERATIONAL  DEFINITION 
OF  TRAINER  ACCEPTANCE 

NATURE  OF  THE  STUDY  DATA 

The  basic  data  deriving  from  the  study  were  in  tho 
following  forms:. 

1.  Numerical  ratings  of  the  judged  adequacy  of  each 
trainer  on  each  of  10  simulation  factors. 

2.  Numerical  ratings  of  the  judged  importance  of 
each  simulation  factor. 

3.  Numerical  ratings  of  12  characteristics  relating 
to  training  effectiveness  and  situational  factors. 

4.  Responses  to  the  questionnaires  designed  to 
amplify  the  information  obtained  from  the 
rating  scales. 

5.  Data  on  trainer  utilization  from  the  NTDC  trainer 
utilization  reports. 

With  respect  to  the  rating  scale  data,  a  numerical  score 
was  assigned  (from  0-15)  representing  the  judgment  of  each 
participant  on  each  characteristic  of  each  trainer.  This  was 
a  simple  one-to-one  linear  translation  of  the  location  of  the 
check  mark  in  relation  to  the  numerical  indices  on  the  scale 
(see  Appendix  A).  These  values  were  averaged  across  all 
respondents . 

With  respect  to  questionnaire  data,  frequency  counts 
were  made  of  the  various  categories  of  response  provided.  In 
the  case  of  open-end  questions,  the  nature  of  the  response 
was  coded  in  terms  of  its  substantive  .content ,  and  whether 
it  was  a  generally  positive  endorsement  of  the  trainer  or 
reflected  a  negative  viewpoint  toward  it.  It  was  determined 
for  various  characteristics  of  each  trainer  whether  the  pre¬ 
ponderance  of  comments  made  was  positive  or  negative  and, 
further,  whether  the  distribution  of  positive  or  negative 
reactions  was  disproportionate  in  relation  to  the  comments 
offered  on  all  trainers  as  a  group. 

Both  the  rating  scale  and  questionnaire  data  represented 
the  subjective  opinions  of  the  users,  although  every  effort 
had  been  made  through  the  design  of  the  data  collection  in¬ 
struments  to  structure  the  judgmental  task  as  carefully  as 
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possible.  It  was  considered  desir.cle,  however,  to  include 
at  least  one  source  of  external  data,  preferably  totally 
objective  in  form,  to  complement  the  opinion  data  in  forming 
a  definition  of  acceptance.  This  requiremeit  was  met  by  the 
inclusion  of  utilization  data  from  the  Training  Device 
Utilization  and  Application  Report  (NAVTRADEVCEN  Report 
10171-4),  which  provided  monthly  data  on  utilization  of  all 
training  devices.  Utilization  data  do  not,  in  themselves, 
constitute  an  adequate  criterion  of  trainer  acceptance. 
Nevertheless,  the>  have  the  merit  of  a  high  degree  of  ob¬ 
jectivity  and  logically  cannot  be  excluded  from  a  compre¬ 
hensive  definition  of  acceptance.  Some  of  their  deficiencies 
are  described  later. 

A  COMPOf*  I  IE  INDEX  '  F  ACCEPTANCE 

In  the  interest  of  convenience  and  to  facilitate  com- 
flunication,  it  xas  desirable  .o  develop  a  single  composite 
index  that  reflected,  a-;  comprehensively  as  possible,  the 
obtained  data  about  each  trainor  that  apparently  reflected 
its  acceptance  by  the  users.  Although  many  approaches  to 
such  an  index  were  possible,  t  was  felt  that  the  index 
should  reflect  the  user’s  ratings  concerning  the  value  and 
comprehensiveness  of  training;  their  judgments  concerning 
the  adequacy  of  simulation;  their  voluntary  comments  about 
the  trainer's  particult"  capabilities  and  limitations  as 
reflected  in  the  questionnaire;  and  the  objective  data 
concerning  actual  utilization. 

A  composite  ''sc.  eocance"  score  was  developed  from  six 
subscores  that  in  "ui",  vere  defined  as  foxlows: 

1.  Felt  valu?  cr  training.  This  score  was  a  simple 
summation  of  r.he  rating  scale  values  assigned  to 
the  trainer  b»  rill  respondents  on  the  following 
f^ve  scales  Crow  the  training  effectiveness 
section, 

a.  Comprehensiveness  of  training. 

b.  Value  of  time  in  the  trainer. 

c.  Use  of  the  trainer  vs.  operational  equipment. 

d.  Value  of  repeated  use  of  the  trainer. 

e.  Comparison  with  similar  trainers. 

The  resulting  scores  for  all  trainers  are  shewn  in  Column  1 
of  Table  3, 
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Simulation  deficiencies.  Since  each  training  device 
was  rated  not  only  on  the  simulation  factors  per  ee 
but  also  on  the  fe'.c  importance  of  each  simulation 
factor,  it  was  possible  to  compare  each  rater's 
judgment  of  the  importance  of  each  factor  to  the 
score  he  assigned  his  particular  trainer  on  that 
factor.  If  the  judgment  of  importance  were  higher 
than  the  score  assigned  to  the  trainer,  this  re¬ 
sulted  in  a  "negative  differential"  score  for  that 
trainer  on  that  factor.  Negative  scores  were  inter¬ 
preted  to  mean  that  in  the  view  of  the  user,  the 
trainer  had  a  deficiency  in  that  area  of  simulation. 
Two  kinds  of  deficiency  indices  were  calculated  for 
each  trainer: 

a.  The  algebraic  sum  of  differential  scores  on  all 
10  simulation  factors; 

b.  The  number  of  simulation  factors  on  which  each 
trainer  received  a  deficiency  score  of  1  rating 
scale  unit  or  more. 

It  was  hoped  that  both  the  severity  of  the  felt 
deficiencies  and  the  generality  of  the  deficiencies 
would  be  reflected  in  the  two  scores.  The  resulting 
scores  are  shown  in  Columns  2  and  3  of  Table  3. 


Voluntary  positive  and  negative  statements  about 
each  trainer .  there  were  seven  open-end  questions 
in  the  questionnaire  which  provided  the  respondents 
with  an  opportunity  to  reflect  favorable  or  un¬ 
favorable  judgments  about  the  trainer. 

Attention  was  focused  on  these  particular  questions, 
not  only  because  of  their  content,  but  because  some 
effort  on  the  respondent's  part  was  required  to 
formulate  his  answer.  For  this  reason  it  was  felt 
that  these  responses  might  be  particularly  relevant 
to  underlying  user  attitudes. 
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b.  As  far  as  you  know,  does  this  trainer  have  any 
kind  of  a  reputation  with  regard  to  maintenance 
or  reliability?  For  instance,  is  it  regarded 
as  a  generally  reliable  or  generally  unreliable 
machine?  I  would  like  to  know  if  the  trainer  has 
a  reputation  in  this  respect,  even  if  it  might 

be  different  from  your  own  opinion. 

c.  To  your  knowledge,  are  there  any  people  now,  or 
have  there  been  any  in  the  past,  with  an  un¬ 
favorable  opinion  of  the  trainer? 

d.  Considering  your  training  needs  and  the  needs  of 
other  trainees,  what  limitations  are  there  in 
what  this  device  was  designed  to  do? 

e.  Overall,  in  terms  of  actually  accomplishing  ef¬ 
fective  training,  what  would  you  say  are  the 
most  eerioua  deficiencies  of  trainers  of  this 
typo? 

f.  What  changes  would  you  suggest  in  order  to  make 
it  easier  or  more  pleasant  tc  use  the  trainer 
without  sacrificing  necessary  realism? 

g.  What  design  changes  would  you  suggest  to  make 
tie  trainer  more  cost-effective? 

For  each  trainer  the  relative  frequencies  of  positive 
and  negative  statements  to  each  question  were  tabu¬ 
lated.  Then,  it  was  determined  whether  tho  distribu¬ 
tion  of  positive  and  negative  statements  about  that 
trainer  differed  substantially  from  the  distribution 
for  all  trainers  as  a  group.  In  cases  where  the 
trainer  received  a  disproportionate  number  of  posi¬ 
tive  01  favorable  comments,  the  trainer  was  giver,  a 
score  of  2  for  that  question.  In  cases  where  a  dis¬ 
proportionate  number  of  unfavorable  responses  was 
recorded  for  a  particular  trainer,  that  trainer  re¬ 
ceived  a  0  for  that  particular  question.  In  the 
event  that  the  responses  were  generally  positive  but 
not  disproportionate  to  those  for  the  total  popula¬ 
tion  of  trainers,  a  score  of  1  was  awarded. 

A  total  score  reflecting  the  reactions  of  all  re¬ 
spondents  to  all  seven  questions  was  then  developed 
for  each  trainer.  The  results  are  shown  in  Column 
4  of  Table  3. 
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4.  Trainer  utilization  data.  As  was  suggested  earlier, 
the  amount  of  utilization  of  a  trainer  may  or  may  not 
be  an  indication  of  its  acceptance  by  the  users. 

Many  administrative  and  practical  constraints  affect 
the  extent  to  which  a  trainer  is  utilized.  A  trainer 
that  is  held  in  high  regard  by  operational  personnel 
may  simply  not  be  scheduled  because  ox  the  urgency  of 
upkeep  responsibilities.  Conversely,  a  trainer  can 
be  heavily  utilized  without  necessarily  being  highly 
accepted  by  its  users  because  it  is  the  only  thing 
available.  For  the  most  part,  however,  high  utiliza¬ 
tion  probably  signifies  substantial  acceptance.  For 
this  reason  it  was  felt  that  utilization  data  should 
comprise  a  part  of  the  composite  index  of  acceptance. 

Data  on  utilization  were  taken  from  a  four-month 
sample  of  NTDC's  records.  The  data  were  selected 
from  the  four-month  reporting  period  immediately 
prior  to  the  time  that  this  study  went  into  the 
field.  In  most  cases,  only  data  covering  the  par¬ 
ticular  trainer  installation  that  the  project  team 
visited  were  utilized.  In  one  or  two  cases,  data  on 
the  same  trainer  at  several  different  locations  had 
to  be  substituted  because  there  was  no  report  from 
the  location  studied.  In  one  case,  the  trainer  in¬ 
stallation  was  too  recent  for  a  data  base  to  have  been 
built  up.  Two  measures  of  utilization  were  used: 
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a.  Percent  utilization.  This  was  defined  as  the 
total  number  of  hours  per  month  the  device  was 
used  for  student  training,  regardless  of  type, 
divided  by  160  hours  (the  number  of  hours  pre¬ 
sumed  to  represent  full  availability  over  a 
working  month).  These  scores  could  exceed  100% 
if  the  device  was  typically  used  cn  more  than 
one  shift.  (See  Column  5,  Table  3.) 

o.  Percent  of  requested  training  accomplished.  Be¬ 
cause  of  the  shortcomings  associated  with  a 
straight  percent  utilization  factor,  it  was  decid¬ 
ed  to  include  a  second  index  from  the  training 
device  utilization  report.  This  was  based  on  the 
ratio  of  the  total  hours  the  device  was  used  for 
training  to  the  total  number  of  training  hours 
requested.  It  was  felt  that  this  index  might  pro 
vide  an  additional  indication  of  the  user's  regard 
for  the  trainer  since  a  trainer  with  a  compara¬ 
tively  low  percentage  utilization  might  neverthe¬ 
less  bo  utilized  a  high  percentage  of  the  time 
the  users  had  available  for  training.  Scores  on 
this  variable  are  shown  in  Column  6  of  Table  3. 
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Tho  next  step  in  developing  the  composite  index  of  ac¬ 
ceptance  was  to  combine  the  scores  for  the  six  separate 
elements  described  above.  In  the  absence  of  any  obvious 
rationale  for  doing  otherwise,  it  was  decided  that  a  simple 
unweighted  composite  of  the  six  scores  would  serve  as  the 
desired  criterion.  However,  since  the  raw  scores  had 
greatly  different  means  and  variances,  it  was  necessary 
first  to  convert  these  to  standard  scores  as  shown  in  Table 
4.  The  algebraic  sum  of  the  standard  scores  was  then  taken 
as  the  composite  index. 

One  other  procedure  needs  to  be  noted.  The  size  of  the 
scores  on  Elements  2  and  3,  in  their  raw  form,  reflected 
negative  viewpoints  toward  the  trainer’s  simulation  charac¬ 
teristics  whereas  in  all  other  cases,  the  higher  the  score 
the  more  positive  the  attitudes  presumably  were.  Conse¬ 
quently,  in  computing  the  sum  of  standard  scores,  the  signs 
for  these  two  variables  were  reversed. 

Table  4  shows  that  the  composite  criterion  scores  ranged 
from  *6. 36  to  -10.11.  The  mean  was  0.7.  For  the  purpose  of 
definition,  trainers  with  standard  scores  of  +3.0  or  uetter 
were  classified  as  generally  highly  accepted;  thsse  with  -3.0 
or  less  were  regarded  as  having  serious  acceptance  problems. 
Table  4  is  useful  for  identifying  the  reasons  for  the  mag¬ 
nitude  of  the  composite  scores.  The  trainers  are  ordered 
in  accordance  with  their  score  on  the  first  element  of  the 
composite  criterion,  i.e.,  the  summation  of  the  five  train¬ 
ing  value  ratings  assigned  by  the  users. 

In  cases  where  similar  trainers  are  involved,  the  source 
of  any  difference  in  their  composite  scores  can  be  determined 
by  inspection.  In  comparing  the  2F66A's  at  San  Diego  and 
Quonset  Point,  for  example,  it  is  evident  that  the  former 
trainer  received  a  somewhat  higher  overall  criterion  score 
than  the  latter.  Inspection  of  the  various  elements  of  the 
criterion  reveals  notable  differences  between  the  trainers  in 
judged  adequacy  of  simulation  and  quite  different  patterns  of 
utilization.  However,  they  received  very  similar  scores  with 
respect  to  positive  recommendations  (Element  4). 

The  2F69A  and  2F69B  offer  another  such  comparison.  In 
this  case,  the  2F69B  received  a  considerably  higher  composite 
score  tl.an  the  2F69A.  However,  it  is  evident  that  most  oi 
this  difference  stems  from  differences  in  utilization.  The 
2F69B,  in  fact,  received  a  lower  overall  rating  on  effective¬ 
ness  factors,  although  it  did  receive  a  somewhat  higher  rating 
on  adequacy  of  simulation  than  did  the  2F69A. 

The  significance  of  these  scores  in  relation  to  the  over¬ 
all  acceptance  profile  for  each  trainer  is  discussed  in  greater 
detail  in  Section  V, 
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INTERCORRELATIONS  OF  THE  ELEMENTS  IN  THE  COMPOSITE  CRITERION 
OF  ACCEPTANCE 

Since  the  various  elements  making  up  the  composite  cri¬ 
terion  reflected  a  mixture  of  relatively  subjective  and 
objective  measures,  it  was  of  interest  to  determine  the  ex¬ 
tent  to  which  they  correlated  with  one  another. '  In  fact, 
the  utilization  data  could  be  considered  as  external  cri¬ 
teria  against  which  the  rating  and  questionnaire  data  might 
be  validated.  Large  correlations  probably  would  not  be 
expected,  however,  because  of  the  previously  mentioned  fal¬ 
lible  nature  of  the  utilization  data.  The  intercorrelations 
of  the  6  elements  are  shown  in  Table  3. 


TABLE  5.  INTERCORRELATIONS  OF  SIX  ELEMENTS 
OF  THE  COMPOSITE  CRITERION  OF  ACCEPTANCE 


Element 

■sniffs 

■3R!RTB 

Element 

1 

■HbMMI 

■vi 

■HI 

HUH 

6 

Element  1 

Tralnlnq  Value 

1  .00 

mm 

-.28 

mm 

Mi 

.54 

Element  2 
Simulation 
Deficiencies 
(Alqebralc) 

-.72 

1  .00 

.60 

-.70 

-.54 

-•71 

Element  3 
Simulation 
Deficiencies 
(Number) 

-.28 

.60 

1 .00 

*> 

-.30 

-.23 

Element  4 

Positive 

Recommendations 

.73 

-.70 

-.58 

1  .00 

.  :o 

.25 

Element  5 

Percent 

Utilization 

.12 

-.54 

-.30 

.10 

1  .00 

.61 

Element  6 

Hours  Used/ 

Hours  Requested 

.54 

-.71 

-.23 

.25 

.61 

1.00 

It  is  evident  that  the  various  elements  derived  from  the 
rating  scale  and  questionnaire  data  correlated  substantially. 
This  was  to  be  expected  because  they  reflected  a  common  source 
of  opinion--namely  that  of  the  users.  It  is  notable,  however, 
that  Element  1  (Training  Value)  correlated  .54  with  Element  6 
(Hours  Used/Hours  Requested)  and  that  Element  2  (Simulation 
Deficiencies)  correlated  -.54  with  Element  5  (Percent  Utili¬ 
zation).  These  are  substantial  relationships  between  very 
independently  derived  measures  and  strengthen  the  conviction 
that  the  various  indices  do  reflect  underlying  attitudes  of 
acceptance . 
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On  the  other  hand,  none  of  the  correlations  was  so  high 
to  suggest  that  any  of  the  elements  was  simply  duplicating 
the  contribution  of  another  in  making  up  the  composite  cri¬ 
terion.  The  highest  correlation  (.73)  was  between  the  judg¬ 
ment  of  Training  Value,  Element  1,  and  the  number  of  dis¬ 
proportionately  positive  statements  volunteered  about  a 
trainer  (Element  4).  The  two  scores  reflecting  utilization 
(Elements  5  and  6)  correlated  only  .61,  indicating  that 
they  do,  to  some  extent,  mean  different  things  with  respect 
to  the  pattern  of  use. 


The 

trainers 

composite  criterion  score  was 
in  the  study  into  three  broad 

used  to  classify  the 
categories : 

1. 

High  acceptance. 

2. 

Moderate  acceptance. 

3. 

Low  acceptance. 

In  the  section  that  follows,  the  individual  trainers 
comprising  these  groups  are  examined  in  detail  and  further 
interpretation  is  made  of  the  probable  significance  of  the 
composite  criterion  score,  and  its  elements,  for  each 
trainer  individually. 
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SECTION  V 

ACCEPTANCE  PROFILES  OF  INDIVIDUAL  TRAINERS 


The  main  results  of  the  study  have  been  summarized  in 
descriptive  profiles  for  each  trainer.  Two  types  of  profiles 
were  prepared,  one  reflecting  the  10  simulation  factors  and 
the  other  the  12  situational  and  training  effectiveness 
factors . 

The  profiles  permit  easy  analysis  of  the  judged  capabil¬ 
ities  and  limitations  of  each  trainer  and  how  each  trainer 
compared  with  all  others  in  the  study.  In  addition,  they 
reflect  how  important  the  participants  considered  a  parti¬ 
cular  feature  to  be  and  whether  the  trainer  measured  up  to 
their  expectations  on  each  factor. 

INTERPRETATION  OF  THE  PROFILE 

The  profile  on  simulation  factors  is  shown  in  two  di¬ 
mensions,  one  boundary  reflecting  the  average  ratings  given 
to  the  trainer  on  each  factor  and  the  other  reflecting  the 
judged  importance  of  that  factor  for  vffective  training. 

In  Figure  2,  a  composite  simulation  profile  for  all  trainers 
is  shown.  The  heavy  dark  line  shows  the  average  values 
assigned  to  the  trainers;  the  shaded  area  represents  the 
amount  of  difference  between  those  values  and  the  judged  im¬ 
portance  of  each  factor  (importance  differentials).  Thus 
the  amount  of  shaded  area  can  be  interpreted  rather  directly 
as  an  indication  of  the  amount  of  felt  deficiencies.  It  is 
clear,  from  Figure  2,  that  the  trainers  as  a  group  were 
felt  to  be  most  deficient  in  simulation  of  vehicle  performance, 
effects  of  controls,  displayed  information,  sensor  performance, 
and  target  reactions.  They  were  generally  in  accord  with  ex¬ 
pectations  in  simulation  of  internal  and  external  operating 
environments,  communication  procedure*,  and  communication 
problems . 

It  will  be  noted  that,  in  general,  the  profile  for  the 
group  of  trainers  varies  about  a  scale  value  of  approximately 
10.  In  designing  the  rating  scales  (M'.pendix  A)  a  scale 
value  of  10  was  defined  as  generally  adequate  or  acceptable. 
Deficiencies  might  be  recognized  but  they  were  not  considered 
so  serious  as  to  detract  significantly  from  the  value  of 
trainer  experience.  Scale  values  of  S  or  lower  were  defined 
as  an  indication  of  an  unacceptable  feature  or  condition. 
Finally,  scale  values  of  15  were  defined  as  reflecting 
clearly  superior  characteristics,  perhaps  (in  the  case  of 
simulation  factors)  being  virtually  non-discriminab le  from 
operating  equipment  or  conditions. 
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Both  the  scale  values  assigned  and  the  importance  dif¬ 
ferentials  should  be  taken  into  account  in  interpreting  the 
profiles.  For  example,  it  is  clear  from  Figure  2  that  most 
trainers  were  rated  low  in  simulation  of  communication  prob¬ 
lems;  however,  the  importance  differential  for  this  factor 
was  quite  small.  Most  users  did  not  consider  this  to  be  a 
serious  deficiency. 

The  composite  profile  for  all  trainers  on  training  ef¬ 
fectiveness  and  situational  factors  is  shown  in  Figure  3. 

It  is  evident  that,  as  a  group,  the  trainers  were  considered 
most  deficient  in  the  areas  of  software  and  completeness  of 
performance  evaluation.  They  received  their  highest  assess¬ 
ments  in  the  opportunity  they  provided  for  supplementing 
operational  experience  and  value  of  repeated  use  (refresher 
training) . 

The  profiles  of  individual  trainers  show  much  greater 
variations  than  the  two  ’'average1'  profiles  presented  in 
Figures  2  and  3.  On  the  following  pages,  profiles  are  shown 
first  for  a  group  of  highly  "accepted"  trainers,  then  for  a 
group  that  clearly  have  acceptance  problems.  Finally, 
trainers  that  are  more  or  less  average  in  acceptance  are 
described.  The  criterion  of  acceptance  was,  cf  course,  the 
composite  standard  score  developed  for  each  trainer  and  shown 
in  Table  4. 
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PROFILES  OF  SOME  HIGHLY  ACCEPTED  TRAINERS 


DEVICE  21C5 :  ADVANCE  SUBMERGED  CONTROL  TRAINER.  The  21C5 
was  designed  to  be  a  "shore-based,  realistic,  high  fidelity, 
dynamic  training  device  capable  of  providing  training  for 
problems  peculiar  to  steering,  diving,  and  casualty  control 
operations  in  STURGEON  class  submarines."  It  was  intended 
to  provide  training  in  the  principles  of  diving,  surfacing 
and  submerged  control;  casualty  and  emergency  operations  that 
might  occur  singly  or  in  combinations;  to  demonstrate  the 
effects  on  ships'  performance  of  various  casualties;  and  to 
accurately  simulate  critical  recovery  factors  and  effects 
such  as  time  required  to  stop  flooding,  the  action  of  planes 
and  rudders,  speed,  main  ballast  tank  blow  and  vent,  negative 
tank  blow  and  vent,  and  low  pressure  blow  system.  The  de¬ 
vice  is  mounted  on  a  platform  which  is  free  to  move  in  pitch 
and  roll.  Movement  in  depth  is  simulated  through  sensor 
displays . 

The  device  is  computer  controlled  in  accordance  with  the 
latest  available  equations  of  motion  for  the  STURGEON  class 
submarine  (SSN637) .  The  simulated  effects  of  environment 
include  surface  effects,  wave  motion  (various  sea  states), 
and  bathythermal  effects.  The  steering  and  diving  stations 
are  faithful  reproductions  of  the  submarine's  interior. 

The  platform  accommodates  three  planesmen,  one  ballast  control 
panel  operator,  and  one  diving  officer.  Simulated  malfunc¬ 
tions  and  flooding  are  controlled  by  the  instructor  by  means 
of  a  keyboard  control. 

The  21C5  is  a  highly  accepted  trainer  with  a  composite 
criterion  score  of  +3.5  (see  Table  4).  The  acceptance  pro¬ 
files  for  the  21C5  are  shown  in  Figures  4  and  5.  With 
respect  to  simulation,  it  is  clear  that  the  design  objective 
of  high  fidelity  simulation  was  clearly  met  in  the  eyes  of 
the  fleet.  The  device  was  rated  uniformly  higher  than 
average  in  virtually  every  respect;  further,  the  discrepan- 
ancies  between  felt  importance  of  various  simulation  factors 
and  their  actual  implementation  in  the  trainer  were,  in 
almost  every  case,  very  small.  Particularly  high  ratings 
were  obtained  with  respect  to  simulation  of  the  internal 
operating  environment,  vehicle  performance,  effects  of  con¬ 
trols,  information  displays,  and  sensor  performance. 

With  respect  to  training  effectiveness  factors,  the 
21C5  also  shows  a  highly  superior  profile.  It  was  rated 
particularly  high  in  problem  setup,  software,  reliability, 
level  of  training,  the  training  opportunities  it  provides, 
value  of  repeated  use,  and  in  overall  comparison  with  si-siiar 
trainers.  Only  in  completeness  of  performance  evaluation  did 
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Figure  5.  Profile  of  Training  Effectiveness  Scores  for  Device  21C5,  Emergency 
Diving  Trainer  (Pearl  Harbor). 
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it  fall  to  approximately  the  average  level  assigned  to  all 
trainers.  This  is  an  interesting  exception  to  the  generally 
very  high  rating  of  this  trainer.  It  may  reflect  a  concern 
expressed  at  the  training  facility  over  the  lack  of  perma¬ 
nently  assigned  instructor  personnel  who  are  qualified  in  sub¬ 
marine  operations.  The  officer-in-charge  indicated  that  the 
availability  of  a  qualified  officer  for  this  billet  would 
greatly  enhance  the  trainer's  value.  The  instructor  operating 
the  trainer  at  the  time  of  our  visit  was  a  highly  motivated 
training  deviceman  who  had  "never  seen  a  submarine."  He  had 
essentially  taught  himself  how  to  instruct  others  in  the  use 
of  the  trainer  through  a  period  of  on-the-job  training.  Un¬ 
doubtedly  he  was  greatly  aided  in  this  process  by  the  Instruc¬ 
tors  Handbook  for  Advanced  Submerged  Control  Training---Device 
21CS  prepared  by  the  Electric  Boat  Division  of  General  Dyna¬ 
mics.  This  publication  provides  25  diagnostic  exercises  with 
detailed  descriptions  of  the  training  objectives,  the  respon¬ 
sibilities  of  the  instructor,  and  the  means  of  problem  setup. 
It  .is  interesting  to  note  that  this  publication  strongly  rec¬ 
ommends  that  the  instructor  be  a  qualified  00D  on  a  nuclear 
powered  submarine  or  a  qualified  engineering  watch  officer. 

The  point  was  emphasized  by  some  submariners  that  it  is 
desirable  to  use  their  own  personnel  as  instructors  because 
of  the  need  to  follow  their  own  particular  standing  orders. 
They  felt  the  instructors  at  the  device  knew  the  basic  opera¬ 
tion  of  the  trainer  well  enough,  but  not  their  own  particular 
operating  procedures. 

"The  instructor  should  be  a  qualified  submariner  so 
that  the  proper  information  is  given  to  the  crews 
at  the  proper  time  and  in  proper  sequence.  As  far 
as  operation  of  the  trainer,  the  instructors  are 
more  than  qualified.  As  far  as  instructing  the  cor¬ 
rect  procedures,  there  are  times  when  it  takes  a 
qualified  submariner  to  know  what  the  causes  and 
effects  are." 

The  instructors  themselves  strongly  felt  their  own  in¬ 
adequacies  : 

"The  operators  are  well-qualified  to  operate  the 
trainer,  but  have  never  been  trained  in  operating 
actual  equipment.  I  don’t  see  how  an  inexperienced 
person  can  instruct  to  the  full  capabilities  of 
the  trainer.  I  have  seen  a  fully  qualified  man 
instruct  and  he  can  cover  so  much  more  it’s  ridic¬ 
ulous  . " 

It  was  felt  by  training  personnel  that  acceptance  of 
the  trainer  would  vary  among  individual  submarine  crews. 


46 


NAVTRADMVCEN  70 -C- 0276-1 


Utilization  had  increased  since  notices  were  sent  out  from 
the  training  center  describing  the  trainer's  capabilities. 
However,  it  was  felt  that  some  submarine  crews  would  not 
use  the  trainer  because  it  did  not  represent  their  esact 
configuration . 

It  was  evident  that  most  users  accepted  the  trainer 
highly,  even  though  they  detected  certain  ways  in  which  it 
was  different  from  actual  equipment.  One  planesman  com¬ 
mented  : 

"It  reacts  a  little  faster  than  our  boat--but  every 
plane  feels  different.  It's  just  a  matter  of 
getting  used  to  the  feel." 

"The  trainer  is  dry.  The  boat  is  not.  You  don't 
get  some  of  the  environmental  effects  (conse¬ 
quences)  --you' Te  not  under  the  same  pressure." 

"The  trainer  acts  a  little  different  from  the  boat. 
Also,  the  gauges  are  set  up  sdmewhat  differently. 

Some  of  the  indications  are  instantaneous ,  whereas 
in  the  boat  the  changes  are  gradual." 

"Background  sounds  are  not  as  high  as  they  are  on 
the  boat--it  lacks  realism  in  this  respect.  You 
notice  every  sound  when  you  have  a  casualty.  For 
example,  the  blow  valve  is  extremely  loud.  We're 
training  mostly  new  people--it's  OK  for  them.  If 
it  were  used  for  refresher  training,  some  of  the 
noises  would  be  desirable.  I  couldn't  talk  to  you 
if  we  had  an  actual  casualty  because  of  the  back¬ 
ground  noise." 

It  was  stated  that  the  trainer  originally  had  beer,  pro- 
graamed  differently  than  it  is  now  and  was  not  acceptable 
until  it  was  programmed  to  more  closely  reflect  the  per¬ 
formance  of  the  637  class  submarine. 

"Originally  the  program  was  designed  toward  what 
they  'thought'  the  637  would  do  on  the  basis  of 
theory.  It  was  necessary  to  change  the  program 
to  reflect  the  operating  characteristics  that  the 
submarine  proved  to  have  with  experience.  There 
were  many  initial  complaints  when  the  trainer 
was  first  installed  for  this  reason.  Operational 
feel  and  handling  are  very  specific  to  a  submarine 
class  and  may  prove  to  b$  different  from  design 
expectations . " 
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MThe  trainer  is  very  good,  even  for  SSN594  class  jg 
submarines.  The  control  room  arrangement  is  j|j 
different  but  the  response  characteristics,  which  ;p 
is  the  important  thing  when  training  on  ship's 

control,  is  very  similar  and  should  prove  to  be  § 
very  useful  for  training."  ‘  I 

9 

However,  the  same  individual,  when  asked  how  he  would  possibly  g 
like  to  see  the  trainer  improved,  could  suggest  only  that  the  *  4 
trainer  be  made  specifically  for  the  SSN595  class.  =§' 


It  will  be  recalled  that  the  participants  in  the  study 
were  divided  into  three  groups  based  on  their  level  of  ex¬ 
perience  in  the  area  of  operations  represented  by  the  trainer. 
The  ratings  of  each  trainer  were  analyzed  separately  for  each 
experience  group  to  determine  whether  there  was  differential 
acceptance  as  a  function  of  felt  "expertise."  The  results 
for  the  21C5  are  shown  in  Table  6.  It  is  evident  that  there 
is  no  clear  relationship  and  that  acceptance  was  high  for  all 
groups.  For  reasons  that  are  not  clear,  the  "medium"  ex¬ 
perience  group  rated  the  trainer  lower  than  the  other  groups 
on  simulation  of  communications,  communication  problems, 
sensor  performance  ar.d  external  operating  environment.  Per¬ 
haps  this  group  is  at  a  level  of  training  that  makes  one 
particularly  sensitive  tc  any  detected  deficiency. 

TABLE  6.  AVERAGE  RATINGS  OF  THE  21C5 
BY  LEVEL  OF  OPERATIONAL  EXPERIENCE  OF  USERS 
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DEVICE  15E16 :  ELECTRONIC  WARFARE  TRAINER.  The  15E16  is  used 
to  teach  all  phases  of  electronic  countermeasures  to  indi¬ 
vidual  operators  and  electronic  warfare  officers.  It  is 
intended  to  train  the  principles  of  operation  of  various  EW 
equipment;  skill  in  the  interpretation  of  displays  and  read¬ 
outs;  skill  in  classifying  intercepted  electromagnetic 
radiations  and  ability  to  analyze  those  radiations  in  detail; 
recognition  of  the  electronic  characteristics  of  various 
types  of  radar  equipment;  and  the  ability  to  determine  the 
priority  of  a  threat  signal.  It  was  expected  that  the 
trainer  would  be  used  by  operators  with  relatively  little 
experience  as  well  as  provide  refresher  training  to  highly 
experienced  personnel. 

The  device  provides  a  means  for  synthetically  generating 
and  distributing  a  number  of  signals  to  each  of  several  user 
ECM  stations.  The  signals  are  supposed  to  simulate  either 
friendly  or  hostile  emitters,  and  are  to  appear  not  as 
"optimum  signals,"  but  as  signals  that  would  be  received 
under  actual  operating  conditions. 

The  qual  fications  considered  necessary  for  instructors 
using  this  device  are  that  they  be  commissioned  officers  or 
senior  petty  officers  with  a  "thorough  knowledge"  of  elec¬ 
tronic  warfare,  of  the  capabilities  and  limitations  of 
Device  15E16,  and  of  the  corresponding  electronic  counter¬ 
measures  equipment  and  procedures  it  is  designed  to  simulate. 

The  15E16  is  a  trainer  that  evidently  had  some  early 
acceptance  problems  but  is  now  highly  accepted  at  Moffett 
Field.  Its  composite  criterion  score  was  +3.5.  Figures  6 
and  7  show  the  acceptance  profiles  for  this  trainer.  With 
respect  to  simulation,  the  15E16  was  rated  distinctly  above 
the  average  of  all  trainers  in  five  areas:  external  en¬ 
vironmental  effects,  equipment  performance,  control  effects, 
display  of  status  information,  and  simulated  sensor  reactions. 
It  received  only  one  below  average  rating,  that  in  the  area 
of  communication  problems.  However,  it  is  important  to  note 
the  high  importance  attributed  to  fidelity  of  simulation, 
particularly  in  equipment  performance,  information  displays, 
and  sensor  performance.  Thus  the  profile  shows  several 
negative  "importance  differentials,"  a  fact  that  very  likely 
somewhat  reduces  its  level  of  acceptance. 

With  respect  to  training  effectiveness  factors,  it  was 
rated  above  the  average  of  all  trainers  as  a  group  on  all 
but  three  factors.  In  fact,  it  received  only  one  below 
average  rating,  on  immediacy  of  performance  evaluation.  It 
was  also  rated  low  on  completeness  of  performance  evaluation 
although  it  did  not  compare  unfavorably  with  other  trainers 
in  this  respect  (Figure  7). 
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The  comparatively  low  rating  of  the  trainer  on  complete¬ 
ness  and  immediacy  of  performance  evaluation  appears  to  stem 
from  the  fact  that  the  instructor  is  unable  to  determine  how 
well  the  student  is  doing  without  getting  up  and  going  to 
each  individual  student  position. 

The  qualifications  of  instructors  are  considered  critical 
in  this  trainer  and  it  was  indicated  that  training  sometimes 
suffers  from  the  unavailability  of  qualified  personnel. 

Although  the  trainer  received  an  above  average  rating  on 
reliability  and  maintainability,  it  is  evident  that  it  is  not 
without  its  difficulties  insofar  as  maintenance  personnel  are 
concerned.  The  trainer  is  "pref lighted"  every  morning  so 
that  any  problems  that  might  arise  are  identified  before  the 
students  arrive.  However,  troubleshooting  is  complicated  by 
incomplete  or  incorrect  documentation;  the  documentation  for 
the  trainer  as  a  whole  is  the  subject  of  considerable  mis¬ 
trust.  Further,  the  modified  GFE  is  not  covered  in  all  the 
documentation.  The  master  scan  generator  is  considered  a 
source  of  particular  difficulty. 

It  was  indicated  that  some  negative  attitudes  developed 
early  toward  this_.trainer  because  of  an  unusually  long  period 
of  time  before  ft  was  accepted. 

"The  allowance  parts  list  did  not  become  available 
until  two  years  after  delivery.  In  the  interim 
several  modifications  were  wade  which  limited  the 
value  of  this  list.  Many  things  were  missing  and 
many  added." 

"Explanations  for  various  systems  in  the  technical 
manuals,  drawings  and  schematics  are  often  too 
brief  considering  the  complexity  of  the  device  and 
the  lack  of  any  formal  maintenance  training  course. 

There  are  too  many  errors  in  the  maintenance 
instruction . " 

Use  of  the  device  is  being  promoted  actively  by  the 
FAETUPAC  and  by  the  instructors  who  are  enthusiastic  about 
it.  The  following  comments  were  offered  by  students: 

"It  is  a  very  good  trainer.  I  learned  a  lot  from 
it.  It's  the  closest  thing  to  being  actually  fly¬ 
ing  that  I  have  seen.  I'm  sure  it's  nothing  like 
flying,  though.  I  do  have  a  few  hours  flight  time 
in  the  radar  seat,  and  that  was  nothing  like  the 
trainer. " 
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"I  had  heard  that  this  was  a  million  dollar  trainer, 
that’ it  was  quite  elaborate  and  supposed  to  he  one 
of  the  best,  around.  I  was  very  impressed  when  I 
first  walked  into  it.  It  made  me  get  the  feeling 
that  I  couldn't  wait  to  use  it." 

The  need  for  training  in  the  recognition  of  individual 
signal  sources  is  clearly  recognized: 

"You  can  never  get  enough  experience  in  the  iden¬ 
tification  of  possible  enemy  emitters.  More  time 
should  be  spent  listening  and  watching  emitter 
signals . " 

A  shortcoming  of  the  device  was  considered  to  be  the 
absence  of  cameras  for  taking  pictures  of  signals  and  re¬ 
corders  for  tape  recording  the  signals: 

"In  the  actual  aircraft  you  must  use  these 
(recorders) . " 

A  point  repeatedly  emphasized  was  the  inability  of  the 
instructor  to  monitor  the  performance  of  six  students  simul¬ 
taneously  and  to  receive  all  of  the  reports  that  they  might 
initiate  for  evaluation.  It  was  felt  that  communication 
between  the  operators  and  the  instructor  was  much  less  im¬ 
mediate  than  in  the  case  of  actual  operations  where  communica¬ 
tions  between  operator  and  TACCO  is  nearly  instantaneous. 

These  criticisms  probably  are  reflected  in  the  somewhat  low 
average  rating  given  the  trainer  in  communication  procedures. 

The  reputation  of  the  trainer  insofar  a^  signal  realism 
is  concerned  was  mixed.  While  some  held  tha;  c  le  trainer 
was  adequate  for  training  in  the  recognition  of  individual 
signal  emitters,  there  were  others  who  felt  "it  is  go^d  for 
learning  operation  of  the  gear,  but  a  waste  of  time  as  far 
as  individual  signal  recognition  goes."  One  problem  men¬ 
tioned  was  the  absence  of  simulated  background  noise  asso¬ 
ciated  with  the  aircraft.  Generators  and  other  equipment 
radiating  electronic  noise  reportedly  complicates  the  prob¬ 
lem  of  ECM  operation  in  the  aircraft  in  a  way  that  is  not 
reflected  in  the  trainer. 

The  number  of  signals  made  available  by  the  trainer  at 
any  one  time  was  considered  to  be  a  limiting  factor.  In 
contrasting  the  15E16  with  the  15E18,  it  was  pointed  out  that 
in  the  latter  trainer  the  computer  has  "every"  signal  avail¬ 
able  and  can  compound  the  problems  that  the  operators  will 
be  expected  to  cope  with  during  an  actual  mission. 
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It  was  also  felt  that  there  was  insufficient  accuracy 
in  target  course,  speed  and  position  information.  This 
added  to  the  difficulty  of  evaluating  student  performance. 

In  addition,  lack  of  provision  for  tape  recording  target 
information  led  to  an  inability  to  demonstrate  the  importance 
of  tapes  to  the  student  and  a  limitation  on  evaluating  his 
performance . 

Several  comments  were  made  concerning  the  lack  of  cor¬ 
respondence  between  the  arrangement  of  equipment  in  the 
trainer  and  that  in  the  aircraft: 

"Designers  made  gear  installation  as  convenient 
rather  than  simulating  real  installations.  It 
doesn't  prevent  good  instruction,  just  realism." 

It  was  felt  that  an  improvement  would  be  to  arrange  the  equip¬ 
ment  in  a  manner  that  it  would  be  identical  to  that  in  the 
aircraft.  It  was  also  suggested  that  all  associated  controls 
should  be  represented  even  though  they  might  have  no  function 
in  the  trainer.  These  included  such  controls  as  the  tape 
recorder  control  box,  ASA-16  marker  panel,  ECM  audio  switch, 
etc. 

In  general,  it  was  felt  that  the  trainer  was  fulfilling 
a  most  important  role: 

"It  gives  the  operator  a  simulated  contact  and  can 
show  his  capabilities  and  limitations.  Without 
the  trainer  the  operator  might  detect  a  contact  in 
the  real  world  and  never  know  what  it  is.  It 
provides  training  in  a  field  the  operator  could 
not  get  otherwise." 

It  was  clear  that  the  instructors  are  playing  a  very 
positive  role  in  publicly  promoting  the  value  of  this  train¬ 
ing  device  and  took  considerable  pride  in  it  being  the  "best 
in  the  training  environment"  at  Moffett  Field.  It  was  gen¬ 
erally  felt  that  use  of  the  trainer  should  be  increased 
through,  for  example,  the  use  of  "mandatory  operator  profi¬ 
ciency  courses"  but  that  there  were  insufficient  instructors 
at  this  time  to  permit  such  usage. 

One  instructor  who  had  over  1200  hours  of  operator  ex¬ 
perience  in  the  P3B ,  most  at  the  ECM  station,  strongly 
emphasized  the  importance  of  electronic  background  noise 
experienced  in  the  aircraft  itself.  The  noise  emanates  from 
the  aircraft  generators  and  other  equipment  and  there  is 
also  a  great  deal  of  weather  noise  in  the  form  of  static 
electricity.  It  was  felt  that  a  provision  should  be  made  to 
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simulate  this  problem  in  the  15E16,  in  the  form  of  a  variable 
noise  control,  so  that  students  "would  know  what  the  real 
world  situation  would  be  like  and  get  used  to  analyzing  sig¬ 
nals  under  adverse  conditions." 

It  was  evident  that  the  15E16  at  one  time  had  a  very 
poor  reputation  with  respect  to  reliability.  However  the 
trainer  at  Moffett  Field  has  been  kept  in  an  excellent  state 
of  repair  by  the  maintenance  technicians.  Nonetheless,  there 
were  numerous  complaints  about  the  difficulty  of  conducting 
maintenance,  performing  calibrations  and  alignments,  and  the 
modifications  necessary  in  the  trainer  to  accept  government 
furnished  equipment.  Schematics  and  interconnect  diagrams 
were  reportedly  extremely  difficult  to  follow  from  system  to 
system  and  there  is  an  absence  of  overall  system  block  or 
flow  diagrams.  Explanations  in  the  maintenance  textbooks 
were  considered  too  brief  in  relation  to  the  complexity  of 
the  device  and  the  absence  of  any  formal  maintenance  training 
course.  Maintenance  instructions  were  reportedly  replete 
with  errors.  Thus,  though  the  equipment  was  generally  regarded 
as  reliable,  it  was  also  regarded  as  very  difficult  to  trouble¬ 
shoot  . 

Evidently,  the  trainer's  reputation  with  respect  to  poor 
reliability  stemmed  from  an  installation  on  the  Atlantic  Coast 
where,  following  a  major  modification  of  the  device,  main¬ 
tenance  personnel  found  themselves  "insufficiently  qualified 
due  to  a  lack  of  interest  and/or  participation  in  the  modifi¬ 
cation."  Consequently,  trainer  reliability  proved  very  poor. 

It  is  evident  that  the  FAETUPAC  Detachment  at  Moffett 
Field  is  doing  a  systematic  promotional  job  with  respect  to 
this  trainer.  It  was  reported  that  they  have  held  open 
house  and  invited  squadron  commanding  officers  and  electronic 
warfare  officers  to  a  presentation  on  the  capabilities  and 
limitations  of  the  device.  This  has  led  to  a  renewed  interest 
at  the  squadron  level  and  higher  morale  among  the  instructors 
who  teach  ECM,  It  has  also  helped  overcome  an  initial  nega¬ 
tive  viewpoint  associated  with  the  eleven-month  period  of  time 
prior  to  acceptance  of  the  trainer.  During  this  time  NTDC 
reportedly  instructed  the  technicians  not  to  get  involved  with 
the  equipment  because  it  was  felt  the  contractor  might  use 
this  as  an  excuse  for  subsequent  system  failure. 

Table  7  shows  that  there  were  no  systematic  differences 
in  acceptance  of  the  15E16  associated  with  amount  of  opera¬ 
tional  experience.  However,  experienced  personnel  rated  it 
much  lower  than  inexperienced  personnel  with  respect  to  the 
level  of  training  for  which  it  is  suited. 
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TABLE  7.  AVERAGE  RATINGS  OF  15E16  BY  LEVEL 
OF  OPERATIONAL  EXPERIENCE  OF  USERS 

Experience  Level 

|  Cow  |  Medium  f 

N=3 


Average  Rating, 
Simulation  Factors 

10.4 

Average  Rating, 

Training  and 

Situational  Factors 

10.4 

DEVICE  14A2:  ASW  TEAM  TRAINER.  The  14A2  is  a  heavily  uti¬ 
lized  team  trainer  for  surface  ship  ASW  operations.  It  has 
simulated  spaces  for  the  bridge,  CIC,  sonar,  and  weapons 
control  aboard  typical  destroyers.  It  is  equipped  with  sim¬ 
ulated  AN/SQS-23  sonar,  MK  111  and  114  fire-control  systems, 
fire-control  radar,  weapon  control  and  launcher  control  sta¬ 
tions,  and  a  target  control  station  manned  by  the  trainer 
operator.  The  entire  complex  is  driven  by  a  general  purpose 
digital  computer. 

The  14A2  is  one  of  the  most  highly  accepted  trainers  in 
the  Navy.  It  received  a  composite  criterion  score  of  6.4, 
the  highest  of  any  trainer  in  the  study. 

The  simulation  profile  for  the  14A2  is  shown  in  Fig¬ 
ure  8.  It  is  evident  that  the  trainer  is  considered  superior 
with  respect  to  simulated  performance  of  the  equipment  and 
vehicle,  the  effects  of  controls,  communication  procedures, 
and  weapon  control.  It  was  considered  about  average,  al¬ 
though  somewhat  below  judged  importance,  in  display  of  status 
information,  sensor  performance,  and  target  reactions.  The 
size  of  these  discrepancies  was  comparatively  small,  however, 
so  that  overall  the  trainer  received  positive  standard  scores 
on  simulation  factors  (see  Table  4). 

The  14A2  was  rated  consistently  above  average  on  train¬ 
ing  effectiveness  factors.  This  was  particularly  true  in 
the  areas  of  software  and  performance  evaluation  (see  Fig¬ 
ure  9).  It  is  believed  that  these  superior  ratings  reflect 
an  unusual  investment  in  the  development  of  utilization 


NAVTRADEVCEN  70-C-0276-1 


guides  and  methods  of  performance  appraisal  associated  with 
this  trainer.*  In  fact,  in  interviews  with  personnel  op¬ 
erating  the  14A2F  (Pearl  Harbor) ,  the  officer  in  charge 
stated  that,,  "we  would  have  been  lost  without  it  (the  uti¬ 
lization  guide)  at  the  beginning." 

Some  observations  with  respect  to  the  simulation  pro¬ 
file  appear  in  order.  Performance  simulation  was  rated  very 
high,  in  part,  perhaps  because  the  14A2  equipment  actually 
operates  "better  than  operational  equipment."  It  is  more 
accurate,  more  effective  (fewer  missed  targets--better 
range--no  detection  uncertainty),  and  suffers  no  equipment 
casualties.  (If  a  casualty  occurs,  the  exercise  is  inter¬ 
rupted  until  it  is  fixed.) 

Although  the  majority  of  respondents  rated  control  sim¬ 
ulation  very  highly,  it  is  interesting  that  the  sonar 
technicians,  as  a  group,  disagreed  with  the  overall  judgment. 
That  is,  they  did  not  think  simulation  was  good  enough. 

This  reflects  their  appreciation  of  the  problems  of  operating 
techniques  in  the  real  world.  Several  operators  commented  on 
the  lack  of  provision  for  developing  contact  classification 
information.  This  criticism  is  also  reflected  in  the  com¬ 
paratively  low  rating  awarded  to  sensor  simulation  for  this 
trainer.  It  is  ironic,  perhaps,  that  some  of  the  acceptance 
of  the  14A2  certainly  stems  from  the  fact  that  sonar  simula¬ 
tion  was  much  more  sophisticated  in  this  trainer  than  in 
earlier  attack  teachers.  Nevertheless,  the  fleet  has  come 
to  recognize  the  importance  of  high  fidelity  sonar  simulation, 
i.e.,  classification  quality. 

In  addition  to  the  "low  quality"  of  sonar  simulation, 
it  is  perhaps  useful  to  point  out  that  no  attempt  was  made 
to  provide  realistic  radar  simulation.  The  complaint  reg¬ 
istered  most  often  in  relation  to  radar  simulation  was  that 
it  made  ASW  air  control  too  easy. 

The  ratings  with  respect  to  target  reactions  probably 
would  have  been  higher  except  that  some  instructors  still 
"play"  with  the  student  since  target  movement  is  directly 
under  his  control.  A  related  shortcoming  was  felt  to  be 
lack  of  realistic  countermeasure  devices,  beacons,  false  tar¬ 
get  cans,  decoys,  etc. 

The  high  rating  on  weapons  firing  and  control  also 
deserves  comment.  A  major  objective  in  the  development  of 

See  Device  14A2  (Series)  Utilization  Guide  for  Surface  Ship 
ASW  Attack  Team  Training ,  Dunlap  and  Associates,  May  1969. 
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the  14A2  was  to  provide  realistic  training  in  the  firing  of 
ASROC  weapons.  The  rating  on  this  item  reflects  the  achieve¬ 
ment  of  that  goal.  Some  dissatisfaction  was  expressed  with 
the  fact  that  the  trainer  has  no  provision  for  torpedo  kill 
probability,  even  though  "rot  all  well-placed  torpedcs  result 
in  hits  and  not  all  torpedus  run  hot,  straight,  and  normal." 

The  high  ratings  on  completeness  and  immediacy  of  perfor¬ 
mance  evaluation  probably  reflect  the  somewhat  special  situa¬ 
tion  where  a  complete  evaluation  team  of  six  or  eight  people 
is  available.  It  is  felt  that  the  ratings  would  have  been 
much  lower  if  they  reflected  the  more  typical  situation  where 
only  a  single  instructor,  seated  at  a  remote  station,  is 
prepared  to  offer  post-exercise  evaluative  commentary. 

It  is  believed  that  the  comparatively  low  rating  on  value 
of  time  in  the  trainer  probably  reflects  a  number  of  considera¬ 
tions.  The  lack  of  high  fidelity  sonar  simulation  has  already 
been  mentioned  and  probably  contributed  to  this  rating.  In 
addition,  any  simulated  ASW  exercise  of  necessity  involves  a 
certain  amount  of  time  during  which  nothing  of  great  signifi¬ 
cance  seems  to  be  happening.  Although  this  is  a  necessary 
element  in  realistic  search  plans  and  contact  developments,  it 
is  possible  that  this  is  construed  as  time  wasted  by  some 
personnel.  Finally,  and  perhaps  most  important,  some  personnel 
reportedly  felt  tnat  the  exercises  in  the  trainer  do  not  ad¬ 
vance  to  a  very  high  level  of  complexity.  Interestingly,  the 
instructors  sometimes  claim  the  students  are  not  ready;  the 
students  say  the  instructors  cannot,  or  do  not  want  to,  develop 
highly  complex  problems. 

Despite  these  identifiable  problems,  in  general  it  is  con¬ 
cluded  that  the  14A2  represents  one  of  the  most  highly  accepted 
and  certainly  most  highly  utilized  training  devices  in  the  Navy. 

There  were  no  systematic  differences  in  level  of  acceptance 
as  a  function  of  the  user’s  level  of  operational  experience. 


TABLE  8.  AVERAGE  RATINGS  OF  14A2  BY  LEVEL 
OF  OPERATIONAL  EXPERIENCE  OF  USERS 

Experience  Level 


Low 

(M=5) 

Medl um 
(N*13) 

HTgn 

(N*12) 

Average  Rating, 
Simulation  Factors 

1  !  .6 

10.6 

10.6 

Average  Rating, 

Training  and 

Situational  Factors 

1  1  .  1 

I  1  .0 

1  1  .6 
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DEVICE  21A38 :  SUBMARINE  ATTACK  TRAINER.  The  21A38  is  a 
shore-based  attack  trainer  comprised  of  three  independent 
attack  centers  that  share  a  common  computer  facility  for 
problem  generation  and  evaluation.  The  trainer  is  designed 
for  the  indoctrination  and  training  of  submarine  fire  con¬ 
trol  teams  in  (1)  the  basic  characteristics  and  capability 
of  fire-control  techniques  and  equipments  utilized  in  sub¬ 
marine  ASW;  (2)  basic  techniques  involved  in  the  approach 
and  attack  of  single  and  escorted  surface  targets;  (3)  ad¬ 
vanced  tactical  training  in  both  pack  and  submarine  versus 
submarine  techniques;  (4)  the  setup  and  firing  of  submarine 
weapons;  (5)  the  integration  of  sonar  equipment  and  operating 
personnel  into  the  fire-control  team;  and  (6)  the  possible 
development  of  new  tactical  doctrine. 

The  acceptance  profile  of  the  21A38  is  complicated  by 
the  fact  that  it  is  essentially  three  trainers  in  one. 

Attack  Center  No.  1  is  designed  to  simulate  modern  diesel 
submarines  and  early  nuclear  submarines.  It  is  equipped 
with  corresponding  sensor  and  fire-control  equipment  (BQS-4, 
EQR-2B,  BQG-4,  and  MK  101  Fire-Control  System). 

Attack  Center  No.  2  is  designed  to  simulate  the  fast 
attack  submarine.  It  is  equipped  with  the  relatively  more 
modern  BQS-6B ,  BQR-7B ,  and  BQH-2C  sonar  systems. 

Attack  Center  No.  3  is  designed  to  simulate  the  ballistic 
missile  submarine.  It  is  equipped  with  the  BQS-4,  BQR-7B,  and 
BQR-2B  sonars  and  the  MK  113  Mod  7  fire-control  system. 

A  difference  of  fundamental  importance  between  the  three 
attack  centers  is  the  presence  of  a  periscope  simulator  which 
is  a  feature  only  of  Attack  Center  No.  2.  This  has  been  a 
major  factor  in  acceptance  as  reflected  by  steadily  increasing 
utilization  of  Attack  Center  No.  2  since  the  periscope  instal¬ 
lation,  and  corresponding  decreases  in  utilization  of  the 
other  attack  centers.  Because  it  was  reported  to  us  that  the 
periscope  simulator  was  a  major  consideration  in  acceptance, 
and  because  of  the  relatively  few  available  users  of  Attack 
Centers  1  and  3,  the  majority  of  the  acceptance  data  gathered 
during  this  study  reflects  user  appraisals  of  Attack  Center 
No.  2.  This  portion  of  the  device  is  highly  accepted  with  a 
composite  criterion  score  of  +5.0. 

The  simulation  profile  for  Device  21A38  is  shown  in  Fig¬ 
ure  10.  The  device  is  rated  as  superior  in  its  simulation  of 
the  internal  operating  environment,  performance  of  the  equip¬ 
ment  and  vehicle,  effects  of  controls,  and  weapon  firing  and 
control.  Felt  simulation  deficiencies  are  reflected  in  the 
display  of  status  information,  sensor  performance,  and  target 
reactions . 
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Profile  of  Simulation  Scores  for-  Device  21A38,  Submarine 
Attack  Trainer,  Pearl  Harbor. 
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In  addition  to  the  periscope  requirement,  the  most  fre¬ 
quently  mentioned  deficiency  of  the  21A38  has  been  the  lack 
of  realistic  sonar  simulation.  Numerous  complaints  were 
registered  by  fleet  personnel  concerning  the  quality  and 
variety  of  target  sounds.  Despite  the  critical  role  of  the 
submarine  sonar  operator  wxth  respect  to  target  detection, 
classification,  and  motion  analysis,  personnel  with  train¬ 
ing  responsibility  aboard  some  submarines  consider  it  a 
waste  of  time  to  send  their  sonar  operators  along  with  the 
attack  team  to  the  21A38.  In  addition  to  poor  audio  simula¬ 
tion,  the  absence  of  a  LOFAR  type  display  is  a  source  of 
complaints.  Few  of  the  operational  problems  encountered  at 
sea  are  well  simulated.  The  result  is  that  the  sonar  opera¬ 
tors  feel  they  play  only  a  minor  role  in  the  trainer  and, 
once  in  the  "automatic  target  following"  mode,  are  called 
upon  to  do  little  but  sit  in  their  chairs. 

The  comparatively  low  rating  on  sensor  simulation  also 
reflects  dissatisfaction  with  the  periscope.  The  quality 
of  target  imagery  was  felt  by  some  to  be  not  as  good  as  that 
in  some  earlier  attack  trainers.  This  makes  "target  iden¬ 
tification  more  difficult  that  it  is  in  the  real  world." 

The  training  effectiveness  profile  of  the  21A38  is  shown 
in  Figure  11.  It  is  clear  that  the  trainer  was  rated  par¬ 
ticularly  high  in  problem  setup,  reliability,  and  level  of 
training;  it  was  also  above  average  in  most  other  respects. 

The  only  below  average  ratings  of  the  21A38  occurred  in 
the  areas  of  performance  evaluation  and  value  of  time  in 
the  trainer.  There  is  an  administrative  problem  that  is 
probably  responsible  for  these  reactions.  There  is  no  stan¬ 
dard  syllabus  or  graded  problem  series  through  which  submarine 
attack  teams  are  regularly  exercised.  Rather,  the  facility 
tries  to  respond  with  the  particular  types  of  problems  that 
a  given  team  may  request.  While  these  requests  almost  always 
can  be  accommodated  in  a  general  way,  the  burden  of  evalua¬ 
tion  and  critique  often  falls  on  enlisted  instructor  person¬ 
nel  who  are  not  fully  qualified  to  assume  these  roles.  This 
stems  in  part  from  their  lack  of  complete  technical  knowledge 
about  all  stations  in  the  trainer,  in  part  from  their  lack  of 
detailed  information  about  various  types  of  targets,  and  in 
part  from  the  social  difficulties  of  critizing  one  of  higher 
rank  than  oneself. 

It  is  interesting  to  note  that  although  the  trainer  has 
a  capability  for  submarine  versus  submarine  action,  with 
different  submarine  teams  operating  in  the  different  attack 
centers,  the  trainer  has  rarely  been  used  in  this  mode.  In 
one  exceptional  case,  two  submarine  teams  were  pitted  against 
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one  another  without  the  knowledge  of  the  commanding  officers. 
The  negative  reaction  of  the  commander  whose  submarine  lost 
the  contest  has  evidently  created  a  lasting  attitude  of 
rejection  insofar  as  this  particular  feature  of  the  trainer 
is  concerned. 

There  is  a  general  feeling  at  the  training  center  that 
qualified  observers  at  the  squadron  level  should  participate 
more  extensively  in  performance  evaluation  in  the  trainer. 

It  was  asserted  that  use  of  the  trainer  is  diagnostic  in  that, 
after  one  or  two  runs,  a  qualified  observer  can  readily  deter¬ 
mine  what  the  deficiencies  of  that  crew  are.  Further,  pre¬ 
dictions  can  be  made  concerning  what  types  of  difficulties 
the  crew  will  have,  given  a  particular  tactical  situation. 

In  summary,  despite  some  notable  deficiencies  in  the 
minds  of  the  users,  Device  21A38  must  be  regarded  as  a 
highly  accepted  trainer.  Personnel  at  the  training  facility 
are  actively  supporting  its  use  by  the  fleet  and  fleet  accep¬ 
tance  appears  to  be  generally  high.  To  the  extent  that  there 
are  negative  reactions,  they  are  associated  most  strongly 
with  inadequate  sonar  and  periscope  simulation  and  with 
deficiencies  in  performance  evaluation  and  supporting  soft¬ 
ware.  There  were  no  significant  trends  in  acceptance  as  a 
function  of  experience  level  (see  Table  9). 

TABLE  9.  AVERAGE  RATINGS  OF  THE  21A38  BY  LEVEL 
OF  OPERATIONAL  EXPERIENCE  OF  USERS 


Experience  Level 


Low 

(N=6) 

Medium 
( N=4) 

Hi  gh 
(N-15) 

Average  Rating, 
Simulation  Factors 

10.3 

10.2 

10.7 

Average  Rating, 

Training  and 

Situational  Factors 

11.4 

9.6 

10.8 

TRAINERS  WITH  ACCEPTANCE  PROBLEMS 


It  has  been  emphasized  that  probably  no  trainer  repre¬ 
sents  a  case  of  complete  acceptance  or  rejection.  Each 
trainer,  whether  generally  accepted  or  not,  may  have  its  strong 
and  weak  points  in  the  minds  of  the  users.  However,  a  number 
of  trainers  did  show  evidence  of  lack  of  acceptance  with  re¬ 
spect  to  many  if  not  the  majority  of  the  characteristics 
studied.  To  emphasize  the  contrast  between  "accepted"  and 
"non-accepted"  trainers,  a  number  of  these  will  be  described 
here . 
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DEVICE  2F55B:  F4B  CREW  TRAINER.  The  2F55B  was  designed  to 
provide  training  in  aircraft  control,  instrument  procedures, 
engine  control,  emergency  procedures,  ECM  operations,  and 
coordination  between  pilot  and  RIO  (Radar  Intercept  Officer) 
in  the  F4B  aircraft. 


The  F4B  is  a  supersonic,  -  -place,  twin-engine,  all- 
weather  fighter  aircraft.  It  was  designed  for  intermediate 
and  long-range  high-altitude  intercept  work,  using  the 
APQ-72  and  APA-157  radar  sets,  and  for  intermediate  or  long- 
range  attack  missions  for  conventional  or  special  weapons 
delivery.  The  responsibilities  of  the  RIO  are  to  manage  the 
airborne  missile  system,  communications,  navigation,  and  ECM. 


The  components  of  the  navigation  and  bombing  systems, 
which  are  viewed,  controlled  or  operated  by  flight  personnel, 
were  to  be  "identical  in  appearances  and  operation"  with  the 
actual  equipment.  The  aircraft  is  designed  to  carry  a  vari¬ 
ety  of  missiles  whose  simulation  was  to  accurately  reflect 
flight  characteristics,  /eight,  and  center  of  gravity  changes. 
Simulation  of  launching,  missile  firing,  and  kill  conditions 
was  to  be  provided.  Target  performance  was  to  be  simulated 
on  the  radar  indicators  in  a  "realistic*  fashion.  In  addi¬ 
tion,  ''maximum  implementation  was  to  be  made  of  human  factor 
considerations  in  the  design  of  the  trainee  and  instructor 
stations  to  insure  optimal  training  value." 

Among  the  trainers  designated  as  low  acceptance  cases, 
the  2F55B  was  the  most  marginal.  Its  composite  standard 
score,  -2.7,  was  not  greatly  below  average,  although  it 
received  negative  scores  on  most  of  the  elements  making  up 
the  composite  (see  Table  4). 

The  simulation  profile  for  the  2F55B  (Figure  12)  iden¬ 
tifies  the  felt  deficiencies  in  greater  detail.  It  is  clear 
that  the  trainer  was  rated  below  average  on  about  half  of 
the  simulation  factors  but  particularly  on  performance  of  the 
equipment,  effects  of  controls,  simulation  of  target  reac¬ 
tions,  and  weapon  firing  and  control.  In  addition,  the 
trainer  was  judged  to  be  significantly  deficient,  compared 
to  judged  importance,  in  its  simulation  of  status  displays 
and  sensor  operations. 

With  respect  to  training  effectiveness  factors,  the 
profile  of  the  2F55B  (Figure  13)  shows  felt  deficiencies  in 
the  area  of  software,  level  of  training,  comprehensiveness 
of  training,  the  value  of  repeated  use,  and  in  overall  com¬ 
parison  with  similar  trainers  of  this  type.  To  understand 
this  profile,  the  following  observations  from  the  interviews 
are  offered. 
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It  is  "a  poor  package  of  hardware  and  there  is  poor 
documentation  on  the  software.  The  trainer  will  not  perforin 
as  a  weapons  system  trainer.  The  radar  and  OFT  portions  can¬ 
not  be  operated  as  a  package.  The  OFT  definitely  does  not 
fly  like  the  F-4  aircraft  as  far  as  general  feel  is  concerned. 
The  first  trainer  arrived  shortly  after  the  aircraft  but  was 
not  operational  for  about  eight  months.  The  first  trainer 
was  designed  around  wind  tunnel  data  and  the  aircraft  proved 
to  be  quite  different.  The  trainer  then  had  to  be  returned 
for  rework.  This  trainer  was  particularly  bad  and  the  whole 
program  suffered  because  of  it.  Attitudes  are  just  now 
changing  because  maintenance  personnel  have  really  worked  to 
get  it  back  in  top  shape." 

Although  it  was  felt  that  the  flight  equations  eventually 
designed  into  the  trainer  were  accurate  and  complete,  it 
reportedly  has  been  difficult  to  maintain  the  proper  adjust¬ 
ments  because  of  maintenance  and  the  unavailability  of  parts. 
Because  of  insufficient  air-conditioning,  the  trainer  has  had 
to  be  operated  under  adverse  conditions.  The  packaging  and 
hardware  are  considered  to  be  of  inferior  quality  and  a  major 
cause  of  troubleshooting  problems.  Drawings  and  schematics 
are  reportedly  incomplete;  no  flow  diagrams  were  provided 
with  the  trainer  with  the  result  that  maintenance  people  had 
to  spend  many  hours  developing  them.  Signal  paths  were  re¬ 
portedly  difficult  to  follow. 

While  a  major  drawback  to  the  trainer  is  associated  with 
its  handling  characteristics  and  flying  performance,  it  was 
also  stated  that  the  tactics  portion  does  not  meet  local 
requirements . 

"The  present  trainer  cannot  track  on  multiple 
navigational  aids,  TACAN,  etc.,  without  great  dif¬ 
ficulty  on  the  part  of  the  operator.  Although 
some  RIO  students  feel  that  the  trainer  is  good 
for  knobology,  ECM  review,  and  navigation  review, 
they  tend  to  feel  that  there  is  insufficient  reward 
for  the  time  spent  in  the  trainer.  They  feel  that 
they  often  just  sit  there  while  the  pilot  plays 
with  the  trainer.  They  also  complain  that  the 
radar  cockpit  systems  are  not  compatible  with  the 
current  cockpit  configuration." 

Coupled  with  the  complaints  about  the  hardware,  there 
were  very  serious  reservations  about  the  quality  of  instruc¬ 
tion  associated  with  this  trainer.  It  was  asserted  that  the 
instructors  do  not  take  the  time  to  find  out  what  the  device 
can  and  cannot  do.  Some  of  the  instructors  were  not  con¬ 
sidered  up-to-date  on  emergency  procedures.  Many  students 
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felt  that  instructor  pilots  should  operate  the  trainer  rather 
than  training  devicemen.  It  was  complained  that  unreal  or 
unrelated  emergencies  were  given  to  the  pilot  simultaneously 
and  that  this  generates  a  bad  attitude  toward  use  of  the 
trainer.  Further,  it  was  felt  that  the  instructors  are  often 
not  thoroughly  familiar  with  what  occurs  in  the  air  and  how 
the  aircraft  responds  to  these  occurrences.  The  complaint 
was  summarized  in  the  assertion  that  "the  things  taught  in 
the  trainer  are  too  simple." 

On  the  positive  side  it  was  stated  that,  "combined  with 
a  working  radar  and  fire-control  system,  this  F-4  trainer 
could  be  utilized  extensively  to  great  advantage.  The 
trainer  is  worthwhile  but  by  no  means  is  it  used  to  do  all 
of  the  things  it  was  designed  for.  It  works  adequately  in 
many  ways  but  the  RIO,  in  particular,  is  being  shortchanged." 

It  was  evident  that,  little  promotional  effort  was  being 
made  to  "sell"  the  trainer  to  its  potential  users.  The 
instructors  expressed  a  need  for  more  access  to  the  squadrons 
so  that  they  could  determine  what  was  desired  in  the  way  of 
training.  They  indicated  that,  in  the  absence  of  any  instruc¬ 
tions  requiring  use  of  the  device  as  a  complete  weapons 
systems  trainer,  a  trend  has  set  in  to  use  it  as  an  emergency 
procedures  trainer  only. 

No  standard  of  qualifications  is  set  for  instructor 
personnel  with  the  result  that  there  are  some  pilot  person¬ 
nel  who  are  considered  quite  acceptable  as  instructors  but 
other  non-pilots  who  are  not.  It  was  asserted  that  "senior" 
instructors  have  an  unfavorable  opinion  of  the  trainer  and 
that  this  opinion  has  spread  through  the  chain  of  command 
and  to  their  flying  peers.  There  was,  however,  no  identifi¬ 
able  difference  in  overall  acceptance  as  a  function  of  level 
of  experience  with  the  F4B  (see  Table  10).  But  highly  ex¬ 
perienced  personnel  did  rate  the  trainer  considerably  lower 
with  respect  to  both  value  of  time  spent  in  the  trainer  and 
value  of  repeated  use.  The  same  personnel  rated  it  higher 
than  neophyte  personnel  in  terms  of  completeness  and  im¬ 
mediacy  of  performance  evaluation.  Perhaps  this  was  because 
they  performed  their  own  evaluations. 


NAVTRADEVCEM  70-C-0276-1 


TABLE  10.  AVERAGE  RATINGS  OF  THE  2F55B 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


DEVICE  2F69A :  P-3  AIRCREW  TRAINER.  The  2F69A  was  designed 

to  simulate  the  P-3A  antisubmarine  patrol  aircraft,  its 
associated  sensors  and  weapons  sytems .  It  was  expected  to 
(1)  simulate  the  actual  aircraft  in  both  normal  and  emer¬ 
gency  flight  conditions;  (2)  permit  training  in  the  use  of 
all  radio  navigational  equipment  for  takeoff  and  en  route 
flying,  approach,  and  landing  problems  associated  with 
instrument  flying;  (3)  permit  training  in  the  use  of  all 
tactical  equipment  installed  in  the  actual  aircraft;  and  (4) 
teach  team  coordination,  team  training,  and  ASW  tactics  for 
the  entire  crew  of  the  P-3A. 

It  was  assumed  that  the  pilots,  engineers,  and  operators 
using  the  trainer  would  be  previously  checked  out  in  the 
basic  operating  procedures  for  their  respective  equipments. 
For  this  reason,  the  military  specification  indicated  ,!that 
the  detail  and  realism  often  associated  with  operator 
trainers  is  not  required.  However,  the  device  must  provide 
all  the  inputs  with  sufficient  realism  so  that  the  crew 
members  can  make  the  necessary  tactical  decisions." 

The  level  of  training  associated  with  this  device  was 
described  as  "advanced."  The  device  was  expected  to  "knit 
the  crew  into  a  team  which  will  permit  them  to  operate  the 
aircraft  for  its  mission  purpose.  Through  canned  and  dynam¬ 
ic  problems  they  will  learn  to  work  together,  exchange 
information,  and  search,  detect,  locate  and  attack  simulated 
targets.  In  this  trainer  they  should  reach  a  level  cf  pro¬ 
ficiency  enabling  them  to  conduct  actual  ASW  operations  at 
sea  in  their  aircraft." 
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The  2F69A  has  fallen  well  short  of  these  objectives  in 
the  view  of  many  users.  Its  score  on  the  composite  accep¬ 
tance  criterion  was  -3.5  (see  Table  4).  It  is  evident  from 
the  profile  of  simulation  factors  (Figure  14)  that  there 
were  serious  differences  between  the  trainer  as  designed  and 
the  felt  importance  of  various  simulation  factors.  Deficien¬ 
cies.  are  particularly  evident  in  the  ratings  of  vehicle 
performance,  effects  of  controls,  display  of  information, 
sensor  performance,  target  reactions,  and  weapon  firing  and 
control . 

Notable  deficiencies  are  also  evident  in  the  profile  of 
training  effectiveness  (Figure  15),  Particularly  low  ratings 
were  given  in  problem  setup,  software,  and  reliability.  Below 
average  scores  were  also  given  on  provisions  for  performance 
evaluation.  These  deficiencies  did  not  preclude,  however, 
relatively  high  ratings  with  respect  to  the  value  of  trainer 
use  (in  comparison  to  opportunities  in  the  operatinal  air¬ 
craft)  and  the  value  of  repeated  use  of  the  trainer.  In  the 
commentary  that  follows,  an  attempt  is  made  to  convey  some 
of  the  reasons  given  for  these  quite  different  viewpoints. 

•'The  tactics  portion  of  the  trainer  has  never  been 
used  since  it  arrived  at  Barber’s  Point.  Prior  to 
that  time  it  had  operated  in  a  coupled  mode  for 
about  two  years  at  Moffett.  It  was  good  in  concept 
tut  the  tactics  portion  was  highly  unreliable.  It 
was  down  in  some  respect  during,  every  other  problem, 
requiring  as  much  as  two  maintenance  shifts  to  keep 
it  operating." 

"The  JEZ  simulation  was  always  very  poor,  never 
really  accepted.  It  took  seven  months  for  them 
to  go  through  a  three  months  acceptance  program. 

NTDC  acceptance  checks  were  largely  static:  many 
of  the  tests  met  the  specs  bur  didn't  really 
tell  you  how  it  would  operate  dynamically.  The 
result  is  that  the  14B31  is  used  for  tactical 
training  instead  of  the  2F69A.  This  is  a  very 
simple  trainer  utilizing  manual  inputs  to  the 
JULIE,  JEZEBEL,  and  TACCO  operators." 

Some  '••juadron  personnel  contended  that  they  got  mors 
training  o-v»  with  the  two  sections  of  the  trainer  (OFT  and 
Tactics)  separated. 

"The.,  is  no  need  to  get  the  ’'hole  crew  to  the 
trainer.  We  would,  however,  like  to  use  the 
2F69A  as  a  full  weapons  system  trainer  for  one 
reeson--to  demonstrate  the  qualifie  tions  of  a 
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particular  crew  member  as  a  member  of  a  coor¬ 
dinated  crew.  We  can  do  this  much  more  econom¬ 
ically  in  a  WST .  We  have  23  qualification 
exercises  that  must  be  achieved  by  each  crew. 
We  frequently  can  run  the  necessary  exercises 
with  only  a  partial  crew  in  the  trainer." 


The  software  associated  with  the  trainer  was  the  subject 
of  considerable  criticism.  The  training  guides  were  described 
as  very  sketchy. 


"We  couldn't  train  simply  using  the  guide.  We 
(the  training  devicemen)  learned  to  operate  th 
tactics  portion  by  observing  the  squadron  in¬ 
structor.  The  operator's  guide  we  have  is  at 
the  bottom  of  a  drawer.  The  instructors  gen¬ 
erally  get  acquainted  with  the  trainer's 
capabilities  by  asking  the  operator  'Can  you 
do  this?"' 


Personnel  in  charge  of  the  trainer  had  developed  descrip¬ 
tions  of  all  of  the  classes  of  problems  they  could  generate, 
but  they  still  depended  on  squadron  personnel  to  define  the 
manner  in  which  they  wished  to  use  the  trainer.  The  opinion 
was  strong  that  they  needed  a  qualified  instructor  attached 
to' the  training  group.  The  fact  that  the  training  devicemen 
in  the  training  group  were  not  "NATOPE  qualified"  was  a 
nutter  of  some  concern.  It  was  felt  that  a  syllabus  should 
be  developed  by  qualified  people  and  then  "....laid  on  the 
various  squadron  training  officers.  At  the  moment,  we  don  c 
know  what  these  people  need." 


w 


It  was  evident  that  the  trainer  was  being  used  to  train 
flight  engineers  in  emergency  procedures  more  than  for  any 
other  purpose.  This  was  one  purpose  for  which  the  trainer 
was  felt  to  be  extremely  valuable.  An  attempt  was  made  to 
schedule  trainee  engineers  for  the  same  exercise  they  would 
have  the  following  day  in  the  aircraft.  It  was  felt  that 
they  could  set  up  many  problems  that  cannot  be  practiced  in 
the  aircraft  with  safety.  However,  the  flight  engineers 
complained  that  there  were  many  system  malfunctions  and 
checks  that  they  could  not  set  up.  They  considered  the 
trainer  generally  deficient  in  the  area  of  simulating  fail¬ 
ures  and  malfunctions.  It  was  stated  that  they  frequently 
have  to  "...jerry-rig  or  cut  out  something  in  order  to  get 
the  desired  effects.  It  should  be  possible  just  to  push  a 
button  to  get  the  desired  setup."  A  related  problem  was 
the  inability  to  simulate  sequential  effects:  "If  one  prob¬ 
lem  develops  in  the  real  world,  another  related  problem  is 
very  likely  to  appear  also.  There  is  insufficient  provision 
for  this  type  of  thing  in  the  trainer." 
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It  was  evident  that  the  trainer  was  being  used  a  sub¬ 
stantial  proportion  of  the  time  in  the  absence  of  not  only 
the  tactical  portion  of  the  crew,  but  the  pilots  as  well. 

One  squadron  engineer  stated  that  they  only  have  a  pilot  in 
the  trainer  about  one-third  of  the  time  that  the  engineers 
are  there.  Therefore,  "If  the  autopilot  portion  of  the 
trainer  is  not  working  (and  it  frequently  is  not),  then  we 
cannot  train.  Sometimes  the  flight  engineers  end  up  flying 
the  aircraft  themselves." 

In  summary,  it  appears  that  the  2F69A  has  suffered  ex¬ 
tensively  from  an  inability  to  keep  the  tactical  portion  of 
the  trainer  operational  and,  as  a  direct  consequence,  has 
been  rejected  in  its  designed  role--that  of  a  vehicle  for 
coordinated  training  of  the  entire  crew.  In  addition,  there 
is  concern  over  the  ability  of  the  instructor  personnel  to 
achieve  the  full  potential  of  the  trainer. 

On  the  positive  side,  it  was  stated  that  before  the 
trainer  became  available,  transition  from  another  type  of 
aircraft  to  the  P-3A  was  particularly  difficult.  It  was 
necessary  t.o  learn  all  the  required  procedures  and  sequenc¬ 
ing  in  actual  flight--"a  very  tough  proposition." 

One  training  officer  claimed  that  the  users  simply  do 
not  appreciate  what  they  have  in  the  trainer- -they  expect 
too  much.  The  same  individual  asserted,  however,  that  the 
2F69A  was  a  procedures  trainer  and  that  the  pilots  cannot 
expect  it  to  fly  like  the  real  thing.  Such  a  view  is  hardly 
in  accord  with  the  objectives  set  forth  in  the  military  char 
acteristics  for  the  P-3A  trainer  series.  Certainly  a  device 
limited  to  procedures  training,  whether  it  be  for  pilots, 
flight  engineers,  or  tactical  personnel,  could  be  achieved 
with  far  less  complexity  and  investment  than  the  2F69A.  As 
a  procedures  trainer,  it  may  be  accepted;  as  a  complete 
weapon  systems  trainer,  it  obviously  is  not. 

Unlike  the  other  trainers  discussed  thus  far,  the 
ratings  of  the  2F69A  suggested  a  general  relationship  be¬ 
tween  trainer  acceptance  and  level  of  operational  experience 
Generally,  the  more  experienced  the  participant  was  in  P-7A 
operations,  the  more  critical  he  was  of  the  trainer  (see 
Table  11).  Criticism  of  the  most  experienced  personnel  was 
directed  primarily  toward  vehicle  performance,  displayed 
information,  sensor  performance,  problem  setup,  software, 
reliability,  and  overall  comparison  with  similar  trainers. 
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TABLE  11.  AVERAGE  RATINGS  OF  THE  2F69A 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


Low 

(N=_4) 

Medi urn 
<N*7) 

HI  gh 
(N=7)  . 

Average  Rating, 
Simulation  Factors 

10.6 

10.5 

8.6 

Average  Rating, 
Training  and 
Situational  Factors 

10.2 

9.7 

8.8 

DEVICE  2F69B- -SOME  CONTRASTS  WITH  THE  2F69A.  Very  likely  in 
recognition  of  many  of  the  problems  cited  in  connection  with 
the  tactics  portion  of  the  2F69A,  the  specification  for 
Device  2F69B  required  that  the  trainer  provide  "realistic  and 
accurate  simulation  for  the  search,  detection,  localization, 
and  attack  functions  of  the  P-3B  antisubmarine  warfare  air¬ 
craft."  There  was  to  be  simulated  activation  of  all  controls, 
instruments,  gauges,  recorders,  plotters,  indicators,  control 
panels,  communication  systems,  lighting  systems,  and  the  like 
associated  with  MAD,  JULIE,  JEZ,  ECM,  navigation,  and  the 
tactical  coordinator  station.  All  modes  of  operation  were  to 
be  simulated. 

Provision  was  to  be  made  at  the  instructor's  control 
station  for  the  development  of  targets  having  carefully 
specified  underwater  sound  characteristics  and  signatures, 
emission  characteristics,  mode  of  operation,  performance, 
and  other  target  type  characteristics.  Detailed  specifica¬ 
tions  were  also  provided  for  ocean  environmental  effects 
that  affect  signal  propagation  and  display. 

A  comparison  of  the  acceptance  criteria  for  the  2F69B 
and  2F69A  suggests  that  ?  number  cf  significant  improvements 
were,  in  fact,  achieved.  The  composite  criterion  score  for 
the  2FC9B  was  +2.2  compared  with  -3.5  for  the  2F69A.  In¬ 
spection  of  Table  4  reveals  that  the  2F69B  received  much 
better  scores  on  simulation  deficiencies  than  the  2F69A. 

Unlike  the  2F69A,  it  is  being  used  as  a  complete  weapons  sys¬ 
tem  trainer  and  enjoying  considerably  higher  utilization.  A 
comparison  of  acceptance  profiles  shows  that  the  2F69B  was 
rated  considerably  higher  than  the  2F69A  with  respect  to  the 
display  of  information,  sensor  performance,  target  reactions, 
and  weapon  firing  and  control  (Figure  16).  It  was  also  rated 
somewhat  higher  in  equipraent/vehicle  performance.  It  is 
noteworthy,  however,  that  in  none  of  these  areas  was  the  2F69B 
rated  as  high  as  the  felt  importance  level  of  these  simulation 
factors . 
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In  comparing  ratings  on  training  effectiveness  factors 
(Figure  17),  it  is  evident  that  the  2F69B  is  still  felt  to 
have  serious  deficiencies  in  the  areas  of  software  and  the 
completeness  and  immediacy  of  performance  evaluation.  In 
most  other  respects,  it  is  similar  to  the  2F69A  and  closely 
approximates  the  general  average  for  all  trainers.  Although 
the  2F69B  is  much  more  heavily  utilised  than  the  2F69A,  it 
still  lacks  endorsement  in  important  ways  from  its  users. 

The  following  comments  are  offered  as  illustrative  of  the 
prob lems : 

"Most  instructors  are  familiar  with  aircraft  opera¬ 
tion  but  limited  in  their  knowledge  of  the  capa¬ 
bilities  of  the  training  device.  The  instructors 
are  not  always  well-qualified  and  current  with 
respect  to  tactics.  ASW  tactical  techniques  have 
changed  dramatically  in  the  last  three  years.  The 
basic  design  of  the  trainer  is  satisfactory  for 
the  techniques  of  five  years  ago  but  it  is  out¬ 
dated  today.  If  the  changes  recommended  for  up¬ 
dating  the  trainer  (were  accomplished),  the  atti¬ 
tudes  toward  the  trainer  would  change." 

"Some  instructors  appear  to  lack  knowledge  of  the 
trainer  and  the  exercises  that  an  be  provided  by 
it.  Where  instructor  operators  are  also  mainte¬ 
nance  personnel,  general  experience  has  shown  that 
many  are  not  interested  in  much  except  the  trainer 
upkeep . " 

"Some  instructors  don't  give  the  student  time  to 
learn  a  basic  skill  before  simulating  emergencies." 

"The  trainer  does  not  fly  like  the  real  aircraft. 

It  is  a  good  trainer  as  it  is  used  to  teach  check¬ 
list  procedures  and  so  forth.  Although  it  does 
not  fly  like  the  aircraft,  it  does  not  really  need 
to  do  so  to  accomplish  training  objectives." 

"The  most  serious  deficiency  of  the  trainer  is  in 
JEZ  training,  but  there  is  no  other  device  avail¬ 
able  to  do  it." 

"Simulation  of  underwater  sound  is  also  a  problem. 

Due  to  poor  design,  the  sound  signals  drift  in 
volume,  frequency,  and  everything  else,  and  must 
be  continually  readjusted  to  keep  them  realistic." 

"The  schematics  and  drawings  are  insufficient  to 
familiarize  maintenance  personnel  with  the  functions 
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of  the  trainer.  The  manuals  are  generally  in¬ 
adequate  in  their  explanations  of  the  functions 
of  various  systems.  A  technician  has  to  super¬ 
impose  his  own  judgment  on  the  adjustment  of 
some  of  the  simulation  systems." 

"Important  topics  in  the  manual  such  as  calibra¬ 
tion  procedures  and  setup  procedures,  along  with 
a  detailed  theory  of  operations,  are  omitted. 

Not  all  subsystems  of  the  trainer  are  covered  by 
drawings  and  schematics.  For  example,  special 
weapons,  D  to  A  converters,  LTV  plotter  cabinet, 
and  subsystems  for  the  land  mass  generation  are 
missing. " 

"It  takes  a  long  time  for  someone  to  become  famil¬ 
iar  with  the  overall  configuration  of  the  trainer 
and  how  it  works.  A  high  degree  of  specialization 
is  required  on  the  part  of  the  individual  mainte¬ 
nance  people.  When  TDs  are  transferred  from  one 
trainer  type  to  another,  their  expertise  does  not 
necessarily  carry  over  to  the  new  trainer  to  which 
they  are  assigned." 

"Someone  is  needed  in  the  line  of  communication 
to  report  the  results  of  complaints  and  try  to 
see  that  the  proper  things  are  put  on  the  trainer." 

"There  have  been  instances  when  paper  work  has 
been  submitted  for  a  modification  to  the  trainer 
and  there  has  been  no  word  or  feedback  at  all  on 
the  submission.  A  direct  line  of  communication 
with  the  modification  and  acceptance  group  of 
NTDC  is  needed  to  find  out  what  has  transpired 
for  follow-up." 

"The  OFT  needs  a  complete  change  in  the  aero  system 
computations  to  assure  good  simulation,  A  quality 
assurance  checking  procedure  should  be  set  up  for 
the  trainer,  run  by  the  people  who  fly  the  aircraft. 
This  would  assure  the  quality  of  the  trainer  by 
checking  systems  not  in  use  for  everyday  training. 

A  check  crew  composed  of  utilizing  squadrons  could 
easily  be  set  up  and  be  of  benefit  to  all.  They 
could  learn  of  the  various  trainer  capabilities  and 
(use)  this  information  (to  achieve)  a  higher  degree 
of  learning  in  the  squadrons.  And  the  maintenance 
people  would  know  the  status  of  the  trainer  in 
relation  to  what  the  users  want  and  expect." 
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"The  trainer  provides  good  experience  for  basic 
skill  acquisition  but  needs  improvement  in  actual 
instrumentation.  Some  instruments  change  too 
fast  with  power  or  switch  positioning  and  others 
are  too  slow.  Simulation  of  the  cockpit  is  good 
and  all  items  are  properly  located." 

"In  general,  for  the  purposes  intended,  the  trainer 
is  all  right.  However,  if  it  was  designed  to  fly, 
it  does  not  do  an  adequate  job  in  training  to  do 
this.  If  the  flying  characteristics  could  be  im¬ 
proved,  it  could  help  training  and  flying  the  air¬ 
craft  for  such  things  as  approaches,  takeoffs  and 
landings . " 

"Performance  reports  should  consist  of  a  detailed 
critique  of  the  entire  period  and  of  the  actions  of 
the  individuals  involved  with  the  training.  Opera¬ 
tors  or  maintenance  personnel  assigned  to  a  trainer 
do  not  have  the  means  of  evaluating  a  student  pilot 
or  flight  engineer  because  they  lack  the  specific 
knowledge  of  flying." 

"The  2F69's  reputation  with  flight  crews  alternates 
between  favorable  and  unfavorable  to  a  wide  degree, 
depending  entirely  on  its  use  or  misuse  by  the 
period  instructor." 

"The  trainer  is  limited  only  by  not  having  a  com¬ 
prehensive  program  of  use  that  is  mandatory  for 
everyone . " 

"For  the  tactics  portion,  if  a  kill  is  made  by,  the 
crew  the  operators  think  that  it  was  a  good  flight 
even  if  all  kinds  of  errors  were  made  in  the  tac¬ 
tical  lead-up.  They  are  unqualified  to  instruct 
each  sensor  station." 

It  was  observed  that  there  is  a  good  exchange  of  infor¬ 
mation  between  the  FAETUPAC  Detachment  at  Moffett  Field  and 
the  squadron  users  of  this  device.  It  appears  that  the  heavy' 
utilization  of  the  2F69B  may  be  the  result  of  aggressive 
salesmanship  on  the  part  of  the  commander  of  the  FAETUPAC 
Detachment.  "He  sells  the  device  as  a  trainer,  not  as  a 
simulator."  In  the  opinion  of  some,  this  distinction  is  an 
important  factor  with  respect  to  acceptance.  It  is  specu¬ 
lated  that  military  personnel  are  more  willing  t  accept 
limitations  in  a  training  device  than  in  a  unit  that  is  pur¬ 
ported  to  be  a  full-fledged  simulator. 
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Table  12  shows  that  there  were  essentially  no  substantia 
differences  in  acceptance  of  the  2F69B  as  a  function  of  opera 
tional  experience  in  the  aircraft.  However,  the  most  ex¬ 
perienced  personnel  tended  to  rate  the  trainer  higher  in 
communication  problems,  problem  setup,  software,  and  level 
of  training. 


TABLE  12.  AVERAGE  RATINGS  OF  2F69B 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


Low 

(N=5). 

Medl up 
(N=ll) 

Hi  gh 
(N=14) 

Average  Rating, 
Simulation  Factors 

10.3 

10.8 

10.7 

Average  Rating, 

Training  and 

Situational  Factors 

8.  ! 

9.0 

9.5 

DEVICE  14B35 :  UNIVERSAL  JULIE/JEZEBEL  OPERATOR  TRAINER.  The 
14B35  is  a  portable,  solid-state  ASW  tactical  crew  trainer 
capable  of  injecting  dynamic  simulated  sonar  signals  into  the 
sensor  equipment  aboard  P-3  aircraft.  The  device  is  intended 
to  provide  ASW  crews  with  a  means  of  "increasing  their  levels 
of  proficiency  in  the  detection,  classification,  and  local¬ 
ization  of  targets."  The  14B35  can  be  used  on  board  an  air¬ 
craft  during  flight,  on  the  flight  line,  in  the  hangar,  or 
as  a  "rooftop"  trainer  for  simultaneous  training  with  several 
aircraft . 

The  14B35  was  developed  expressly  "to  improve  fleet  per¬ 
formance  in  the  prosecution  of  JULIE  and  JEZEBEL  contacts, 
to  reduce  the  number  of  false  contacts  and  false  classifica¬ 
tions."  With  these  objectives,  the  military  characteristics 
clearly  reflected  the  need  for  realism  and  sophistication  in 
signal  simulation  "above  that  currently  provided  in  existing 
part-task  training  devices."  Training  was  to  be  directed  at 
the  intermediate  and  advance  levels  and  to  encompass  all 
activities  from  initial  LOFAR  detection  through  final  S0N0- 
BUOY  fixing. 

The  instructors  for  the  device  were  to  be  "highly  quali¬ 
fied  officers  and  senior  petty  officers  with  a  comprehensive 
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background  in  ASW  tactics  and  detection  systems."  It  was 
recommended  that  the  instructor  be  either  a  qualified  ASW 
pilot  or  tactical  coordinator.  All  trainees  were  expected 
to  have  had  basic  instruction  in  the  operation  of  their 
equipment . 

This  device,  which  appears  excellent  in  concept,  re¬ 
ceived  next  to  the  lowest  composite  criterion  score  of  all 
the  trainer*  studied  (-7.6).  It  was  rated  below  average 
on  all  of  th'e  elements  comprising  the  composite  criterion 
(see  Table  4).  On  the  simulation  profile  (Figure  18),  it 
received  exceptionally  low  ratings  on  display  of  information, 
sensor  performance,  and  target  reactions.  It  was  also  rated 
well  below  the  judged  importance  level  on  environmental 
simulation  (presumably  the  effects  of  environment  on  signal 
presentation),  equipment  performance,  effects  of  controls, 
and  communication  problems. 

With  respect  to  training  and  situational  factors,  it 
received  below  average  ratings  on  almost  ail  factors 
(Figure  19),  but  particularly  problem  setup,  completeness 
of  performance  evaluation,  value  of  time  spent  with  the 
trainer,  value  of  the  trainer  versus  operational  equipment, 
and  in  overall  comparison  with  similar  trainers. 

The  lack  of  adequate  submarine  target  services  for  ASW 
training  is  one  of  the  most  widely  recognized  problems  con¬ 
fronting  the  fleet.  It  is  surprising,  ther ,  to  find  such  a 
lack  of  acceptance  of  a  device  that  was  specifically  designed 
to  enable  aircrews  to  use  their  operational  equipment  in  the 
absence  of  submarine  services.  To  determine  why  this  was  so, 
some  of  the  responses  from  the  questionnaires  may  be  reveal¬ 
ing.  The  following  comments  were  offered  by  instructor 
personnel : 

"It  is  a  rather  complex  device  in  some  areas  and 
requires  sophisticated  knowledge  in  regards  to  main¬ 
tenance.  The  transmitter  section  is  definitely 
critical . " 

"The  device  is  limited  in  that  it  does  not  simulate 
DIFAR.  It, is  understood  that  a  modification  is 
under  development  that  will  permit  this  capability." 

"The  device  is  not  used  by  deployed  squadrons  to  its 
full  potential.  Some  instructors  are  well-qualified 
and  some  aren’t.  Some  have  trouble  keeping  up  with 
the  problem.  Operators  with  varying  experience  are 
employed  as  instructors--obvious  ly ,  fleet  people 
have  better  knowledge  than  others." 
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''The  device  is  reasonably  effective  for  basic 
skill  training- -however ,  two  of  these  skills 
will  soon  be  obsolete.  Certainly  it  has  no  use 
for  advance  training  as  it  is  too  dependent  on 
human  inputs." 

"A  deficiency  of  the  trainer  results  from  the 
necessity  of  the  device  operator  having  to  in¬ 
sert  the  tactical  information  manually.  The 
communication  between  the  crew  and  device  opera¬ 
tor  detracts  from  crew  performance  and  problem 
f low . " 

"Due  to  the  manual  dependence,  I  can’t  think  of 
any  way  of  improving  this  trainer  except  to 
allow  for  thirty-one  channel  capabilities  for 
search  and  one  channel  for  localization.  How¬ 
ever,  because  the  device  is  used  for  very  basic 
skills,  it  is  not  critical  that  these  improvements 
be  made.  It  would  probably  cost  the  Navy  more 
money  than  the  effort  is  worth." 

"The  trainer  is  reasonable  for  very  basic  skills. 
The  crew  is  at  least  in  the  air  and  you  have 
aircraft  movement  (in  contrast  to)  the  ground 
trainers.  I  feel  it  worthwhile  in  a  training  en¬ 
vironment  only--!  would  venture  to  say  there 
would  be  little  use  in  an  operational  squadron." 

"Another  deficiency  is  restricted  simulation. 

The  device  provides  some  unrealistic  displays 
but  these  aren’t  serious  enough  to  detract  from 
the  use  of  the  device  for  basic  skills.  Remem¬ 
ber  we  are  talking  basic  skills  and  we  are 
teaching  more  equipment  manipulation  than  pure 
tactics . " 

"The  device  is  limited  in  the  number  of  modes  in 
which  it  can  be  operated  simultaneously.  Some 
tactics  require  that  wc  operate  in  two  modes  at 
the  same  time." 


"The  trainer  has  a  poor  reputation  because  every- 
ons  seems  to  think  it  should  be  just  like  the 
real  thing.  They  can’t  seem  to  accept  the  fact 
that  it  is  a  simulator  instead  of  a  duplicator." 

"New  operators  (acting  as  instructors)  experience 
notable  lag  periods  in  making  the  proper  inserts 
to  the  trainer  when  working  with  more  experienced 
proficient  trainees." 


•Ssz&ti 


NAVTRADEVCEN  70-C-0276-1 


"Some  skills  cannot  be  trained  because  oceano¬ 
graphic  phenomena  cannot  be  introduced." 


"Practical  JEZ  analysis  cannot  be  adequately 
trained  because  of  no  ocean  hindrances  or  simu¬ 
lation  of  such." 


"The  instructions  necessary  for  operations  are 
often  garbled  by  the  inexperienced  student  (90% 
of  our  student  load) ,  thus  hindering  proper  use 
of  the  device.  The  instructions  required  are 
artificial  and  distract  from  good  real-world 
oriented  trai-'ing." 


The  following  comments  were  offered  by  students 


"Equipment  malfunctions  have  limited  the  device 
in  training  hours  accomplished.  Lack  of  knowledge 


by  equipment  operators  also  limits  its  use. 
Bulkiness  and  weight  do  not  make  it  conducive  to 
frequent  use.  On  the  squadron  level  it  is  sched¬ 
uled  very  infrequently  on  the  flight  schedule. 
(Nine  times  out  of  ten  this  is  due  to  a  lack  of 
a  trained  14B35  operator.)" 


"We  never  know  whether  or  not  it  is  going  to  mal¬ 
function.  If  it  does  malfunction,  and  it  does 
frequently,  it  always  happens  in  the  middle  of 
an  ASW  problem.  No  one  really  feels  confident 
that  the  trainer  will  hold  up  for  the  entire  prob¬ 
lem." 


"We  have  had  (trainer)  operators  who  didn't  even 
know  how  to  hook  up  the  14B35  and  were  actually 
asking  the  crew  members  'what  went  where?'" 
(However,  this  same  respondent  felt  that  it  could 
be  a  good  and  valuable  training  aid  if  used 
correctly . ) 


"There  seems  to  be  a  disparity  between  what  the 
equipment  was  designed  to  do  and  what  it  has 
actually  accomplished." 


"There 
at  the 
know  what 


are  deficiencies  (who  knows  how  to  fix  one 
squadron  level?).  If  the  operator  doesn't 
going  on,  what  good  is  the  trainer? 


is 


and  bulkiness  and  weight--when  you  have  to  lug 
this  thing  out  to  the  aircraft  you  think  twice 
before  you  do  it." 
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"TLe  operators  do  not  have  enough  knowledge  of 
tactics  to  fully  operate  the  trainer  to  the  best 
of  its  capabilities."  / 

i 

"By  the  time  the  squadron  was  proficient  with  the 
use  of  the  trainer,  we  had  new  equipment  on  board 
the  aircraft  that  would  not  accept  the  trainer 
outputs." 

"The  JEZ  write-outs  are  def£cie,ht  in  that  there 
are  no  accurate • dynamic  changes,  very  few  target 
operating  modes,  and  no  aural  characteristics." 

t 

"The  trainer  is  very  good  if  /used  to  perfect  crew 
coordination.  But  it  does  not  give  the  different 
crew  members  good  training  i'n  their  area  of 
operation."  / 

"The  trainer  has  a  reputation  for  unreliability 
due  to  damage-  that  occurs/while  moving  the 
trainer  from  aircraft  to  .aircraft  cr  back  to 
stowage  space." 

"The  basic  concept  of  the  trainer  is  right--it 
ju^t  doesn't  work  all  of  the  time." 

"The  trainer  operator  is  required  to  manipulate 
the  trainer  so  fast  on  occasions  that  mistakes 
are  made  by  the  operator  with  low  experience 
levels.  I  think  that  if  some  of  the  controls 
could  be  operated  automatically  by  the  training 
device  a  more  realistic  problem  could  be  experi¬ 
enced  by  the  trainees.  Computerized  inputs  would 
be  highly  desirable." 

"The  real  world  has  more  variables  than  this  trainer 
simulates.  Water  conditions  are  quite  erratic  as 
are  submarine  source  levels." 


"The  school  for  the  instructors  is  too  short.  I've 
been  there.  Too  much  fix  it,  not  enough  operate 
it." 

The  following  comments  were  offered  by  maintenance"' 
personnel : 


"Not  all  computer  logic  symbols  are  explained  in 
the  maintenance  manual.  Not  all  equipment  needed 
for  troubleshooting  is  available.  The  parts  list 
is  incomplete  and  vague  in  the  description  of 
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component  parts.  Many  times  parts  are  not  avail¬ 
able  when  needed.  Maintenance  procedures  in  the 
maintenance  handbook  are  not  always  correct  or 
clear  in  description." 

"Most  of  the  troublesome  problems  have  been 
rectified.  However  the  transmitter  unit  requires 
frequent  alignment  and  the  PDC  switch  on  the  1A1 
console  requires  frequent  replacement." 

"Due  to  the  purpose  or  function  of  this  device,  a 
thorough  knowledge  of  ASW  tactics  and  procedures 
is  required  (by  maintenance  personnel).  Most  TDs 
get  no  ASW  training  of  any  kind  in  "A"  or  "B" 
schools . " 

"I  would  say  that  it  is  easy  to  maintain  and  a  very 
reliable  trainer.  Also  I  would  say  that  the  East 
Coast  FAETUs  have  had  a  very  poor  14B35  program  and 
that  the  West  Coast  could  give  them  valuable  assis¬ 
tance  in  setting  -p  a  good  workable  program." 

"East  Coast  personnel  have  an  unfavorable  view  of 
the  trainer  because  they  were  given  the  14B35 
devices  cold.  They  received  the  device  (I  know 
for  I  was  there  at  the  time)  and  the  manuals  apd 
were  expected  to  operate  it  without  a  school.  ' 

They  therefore  do  not  know  how  to  use  the  device 
with  the  proficiency  that  is  required  to  make 
the  14B35  an  effective  trainer." 

In  summary,  the  14B35  appears  to  be  a  device  that  was 
excellent  in  concept  but  deficient  in  design  for  its  in¬ 
tended  purpose.  The  design  objectives  have  been  frustrated 
both  by  basic  deficiencies  in  simulation  and  by  an  apparent 
failure  by  the  users  to  recognize  the  investment  required 
ir.  training  personnel  for  the  proper  operation  of  a  device 
of  this  kind.  The  fallacy  that  every  operator  is  an  expert 
instructor  was  never  more  clear.  Not  only  is  the  device  not 
accepted  for  its  intended  purpose  of  advanced  training  in 
detection  and  classification,  it  has  perhaps  suffered  a 
worse  fate  by  becoming  thought  of  as  a  "basic  skill"  trainer 

It  is  also  clear  (Table  13)  that  those  personnel  with 
the  greatest  amount  of  operational  experience  are  the  most 
critical  of  the  trainer.  They  rated  it  much  lower  than 
inexperienced  personnel  with  respect  to  displays,  level  of 
training,  the  training  opportunities  it  provided,  and  value 
of  time  spent  using  the  trainer. 
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TABLE  13.  AVERAGE  RATINGS  OF  THE  14B35 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


Low 

(N=4) 

Medi urn 
(N=9) 

High 
(N  =  5) 

Average  Rating, 
Simulation  Factors 

8.2 

9.3 

5.2 

Average  Rating, 
Training  and 
Situational  Factors 

8.5 

8.  1 

6.7 

DEVICE  2H87 :  CARRIER  LANDING  TRAINER.  The  2H87  is  an  air¬ 
craft  carrier  approach  and  landing  trainer.  The  objective 
of  its  development  was  to  provide  a  device  which  could  be 
used  to  train  and  develop  jet  pilots  in  the  skills  and 
procedures  necessary  to  make  safe  carrier  landings.  The 
trainer  was  to  provide  training  in  (1)  cockpit  procedures 
related  to  the  approach  and  landing  situations;  (2)  instru¬ 
ment  flight  procedures  in  the  carrier  vicir.ityi  and  (3) 
procedures  required  for  the  pilot  to  successfully  land 
aboard  a  carrier  under  all  visual  conditions,  day  or  night 
(from  CAVU  to  minimum  visual  conditions). 

It  was  anticipated  that  the  device  would  be  used  in  the 
basic  training  command  by  students  prior  to  their  carrier 
qualifications  phase  or  by  qualified  carrier  pilots  to  main¬ 
tain  proficiency  when  shore  based. 

The  visual  presentation  was  to  be  such  that  it  would 
"realistically  respond  to  the  simulated  aircraft  movements 
and  appear  as  a  carrier  would  to  the  pilot  whenever  it  was 
within  viewing  conditions."  Geometric  distortion  and  jitter 
in  the  display  were  to  be  minimized  to  the  extent  that  they 
^ould  not  influence  trainee  acceptability  of  the  device. 

t 

I  Sufficient  resolution  was  required  in  the  display  so 

/  that  the  Fresnel  simulation  would  provide  usable  glide  slope 
/  information  to  the  trainee  within  one  and  one-half  miles 
from  the  stern  of  the  carrier  under  VFR  conditions. 


The  device  was  to  conform  to  the  T-2B  aircraft  configu¬ 
ration  except  that  the  full,  range  of  T-2B  simulated  performance 
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in  f Hying  qualities  was  not  a  simulation  requirement.  How¬ 
ever,  simuiation  was  to  "realistically  duplicate  the  air¬ 
craft’s  flight  control  system  in  the  landing  configuration, 
including  control  displacement,  forces,  feel  and  aircraft 
frequency  response  characteristics  . "  Simulated  instrument 
responses  were  also  to  duplicate  those  of  the  aircraft  when 
ir.  this  configuration. 

Of  all  the  trainers  included  in  this  study,  the  2H87 
carrier  landing  trainer  enjoyed  the  least  acceptance.  In 
fact,  it  was  not  possible  to  get  meaningful  evaluations  from 
student  personnel  because,  at  the  time  of  the  study,  the 
trainer  was  being  used  only  as  a  means  of  visually  present¬ 
ing  the  simulated  scene  during  a  catapult  shot  and  landing 
as  the  trainees  sat  passively  in  the  cockpit.  They  were 
permitted  no  active  control  of  the  training  device  at  all. 

No  trainees  who  had  experienced  the  trainer  used  in  this 
fashion  were  available  to  us  at  the  time  of  the  investiga¬ 
tion.  However  it  was  possible  to  obtain  ratings  and  inter¬ 
views  with  highly  qualified  administrative,  instructor,  and 
maintenance  personnel.  The  composite  criterion  score 
assigned  by  these  personnel  was  -10.1.  The  results  of  their 
ratings  are  shown  in  Figures  20  and  21. 

With  respect  to  simulation  factors,  it  is  evident  that 
the  trainer  was  rated  extremely  low  on  virtually  every  fac¬ 
tor:  internal  operating  environment,  external  environment, 

vehicle  performance,  controls,  displays,  and  sensors. 

The  trainer  was  also  rated  well  below  average  in  most 
training  effectiveness  factors  but  particularly  problem  set¬ 
up,  software,  comprehensiveness  of  train.ing,  the  value  of 
the  trainer  in  relation  to  operational  equipment,  and  in  the 
value  of  repeated  use  of  the  trainer.  Because  of  the  way 
the  trainer  was  being  used,  no  meaningful  scores  were  obtained 
on  trainer  reliability  or  on  comparison  with  similar  trainers. 
In  the  former  case,  only  one  respondent  provided  any  ratings 
and  it  was  felt  that,  in  its  present  mode  of  use,  valid 
evidence  of  reliability  was  lacking.  In  the  latter  case, 
there  were  really  no  similar  trainers  with  which  the  2H87 
could  be  compared. 

It  was  evident  that  in  the  opinion  of  the  users  the 
trainer  has  major  deficiencies  both  as  a  flight  trainer  and 
with  respect  to  the  visual  presentation  of  the  simulated 
environment.  In  an  effort  to  understand  the  nature  of  these 
felt  deficiencies,  the  following  comments  from  instructor 
and  administrative  personnel  are  presented. 

"The  concept  of  this  device  was  excellent  but  it 

exceeded  the  state-of-the-art  at  the  time  of 
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procurement.  Rapid  turnover  of  project  officers, 
ir.f ite  and  ambiguous  specifications,  improper 
progress  evaluations,  misconception  of  the  purpose 
of  the  device,  and  political  sparring  created 
chaotic  results." 

"Device  acceptance  to  this  day  is  shrouded  with 
misunderstanding.  The  result  was  that  an  un¬ 
suitable  device  was  procured  at  an  exorbitant 
cost,  and  it  cou.ld  n- ■*  be  used  for  student  train¬ 
ing.  It  lay  idle  fo'.  :  yesr.  After  a  suggestion 
that  the  device  be  either  tvciiized  or  turned  over 
as  excess,  the  present  concept  of  use  was  devel¬ 
oped.  It  is  felt  however  that  the  device  will 
soon  be  lost  since  it  is  a  poor  substitute  for  its 
original  concept." 

"The  trainee  cannot  manually  control  this  trainer 
due  to  differences  between  performance  of  the 
trainer  and  the  aircraft.  It  doesn’t  fly  like 
the  aircraft." 

"Instructors  cannot  handle  the  trainer.  It  would 
do  more  damage  than  good  to  the  student  it  its 
present  state.  The  instructors  are  not  qualified 
because  they  are  trainer  operators  not  trainer 
aviators.  Qualified  LSO’s  should  serve  as  in¬ 
structors  or  at  least  oversee  the  flight." 

"Everyone  who  has  flown  the  trainer  has  an  un¬ 
favorable  opinion  of  it.  It  is  too  difficult  to 
control,  and  the  presentation  is  not  good  enough." 

"At  present  the  trainer  is  of  limited  use  for 
students  because  of  its  poor  response  or  feel. 

The  only  value  for  students  is  to  show  them  lim¬ 
ited  approaches  and  catapult  shots.  The  LSO 
instructors  feel  the  trainer  would  give  the  stu¬ 
dent  a  feeling  of  insecurity  if  they  did  poorly 
in  flying  the  trainer.  This  would  (have)  an  un¬ 
desirable  effect  on  their  level  of  confidence 
when  it  is  time  to  go  to  the  ship." 

"Instructors  are  not  well  qualified  to  evaluate 
student  performance  in  the  trainer  because  they 
are  not  LSOs.  Therefore  they  can  only  evaluate 
the  computer  readout  which  is  not  the  same  as 
the  LSOs’  analysis  of  each  approach." 

"Student  personnel  who  have  flown  the  2H87  feel 
that  the  inputs  generated  by  the  student  are 
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overcontrolled  by  the  computer  inputs,  thereby 
giving  a  false  and  unrealistic  picture  of  an 
actual  carrier  approach.  They  also  feel  that 
visual  resolution  is  very  poor." 

"I  would  like,  for  once,  to  see  the  Navy  utilize 
the  opinions  of  the  user  and  not  rely  so  heavily 
on  engineers  and  computer  specialists  who  have 
never  seen  a  carrier  approach  and  landing  in 
developing  a  trainer  such  as  this  one.  If  it 
would  have  been  proper?/  engineered,  this  trainer 
could  have  been  a  tremendous  aid  in  training 
students . " 

"As  far  back  as  1967,  VT4  sent  pilots  to  the  con¬ 
tractor's  plants  to  help  the  engineers  work  out 
the  problem  areas.  My  pilots  reported  to  me  when 
they  returned  from  seeing  the  mock-up  that  such 
information  as  glide  slope  and  so  on  were  all 
wrong.  This  problem  was  a  consequence  of  not 
having  an  LSO,  who  knew  what  these  parameters 
should  be,  serving  as  a  project  manager.  It  was 
no  fault  of  the  contractor;  they  were  only  going 
on  what  specifications  were  given  to  them  by 
NTDC . " 

"A  new  high  resolution  camera  that  is  now  avail¬ 
able  through  advancement  of  the  state-of-the-art 
would  increase  the  use  of  the  trainer  99%.  Re¬ 
programming  the  trainer  to  fly  in  accordance 
with  the  aircraft  is  a  relatively  simple  matter 
I  am  told.  The  trainer  would  be  extremely 
valuable  to  this  command  if  it  worked  as  ad¬ 
vertised  . " 

In  summary,  the  whole  history  of  this  device  is  cr.e  of 
confusion  and  concern.  NTDC  developed  the  specifications  for 
a  prototype  device  with  the  apparent  objective  of  making  it 
versatile  enough  to  evaluate  a  variety  of  visual  systems 
that  might  be  utilized.  In  fact,  it  was  built  with  a  rather 
inflexible  visual  system  th<zt  is  more  complex  than  necessary. 

In  many  respects  the  engineering  of  the  trainer  seemed  to 
be  very  good  and  the  concept  certainly  was  without  fault. 

However,  the  failure  to  adequately  simulate  the  performance 
of  the  T2  aircraft  seems  to  have  been  an  overwhelming  in¬ 
fluence  that  precluded  trainer  acceptance  for  any  purpose. 

Nevertheless,  an  interesting  feeling  exists  concerning 
the  value  of  the  2H87.  With  all  its  admitted  faults,  some 
people  feel  that  the  programmed  visual  catapult  shots  have 
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a  positive  training  value.  "Students  can  at  least  be  given 
a  presentation  of  how  rapidly  things  change  during  catapult 
takeoffs,"  This  feeling  is  related  to  an  incident  on  a  car¬ 
rier  where  a  student  pilot  was  involved  in  a  bridle  breaking 
during  run-up.  When  the  bridle  parted,  he  thought  the  cata¬ 
pult  had  been  fired  and  let  the  airplane  roll.  Subsequently 
he  went  over  the  side  and  was  killed.  Everyone  associated 
with  the  trainer  relates  this  story  as  an  example  of  how 
effective  the  trainer  could  be  in  at  least  acquainting  the 
student  with  the  visual  changes  associated  with  catapult 
operations . 

Because  of  the  nature  and  small  number  of  qualified 
respondents,  the  relationship  between  operational  experience 
and  trainer  acceptance  cannot  be  described  with  confidence. 
It  is  doubtful  whether  any  significant  relationship  could  be 
shown  in  the  absence  of  student  pilot  opinion.  However,  the 
data  are  shown  in  Table  14  in  the  interest  of  completeness. 
Experienced  personnel  tended  to  be  particularly  critical  of 
the  environmental  simulation,  vehicle  performance,  and 
control  effects. 

TABLE  14.  AVERAGE  RATINGS  OF  THE  2H87 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


(N=2) 

Medl um 
(N=1) 

HTgfi 

(H-5).. ...... 

Average  Rating, 
Simulation  Factors 

5.6 

6.6 

5.  i 

Average  Rating, 

Training  and 

Situational  Factors 

4.5 

10.2 

8.4 

PROFILES  OF  THE  REMAINING  TRAINERS 

The  remaining  trailers  in  the  sample  were  assigned  mid¬ 
range  values  on  the  composite  criterion  of  acceptance, 
representing  neither  out'  .ariding  examples  of  acceptance  nor 
rejection.  They  did  differ  substantially  in  acceptance 
with  respect  to  particular  factors,  however.  Their  profiles 
will  be  presented  in  the  sections  that  follow  roughly  in 
the  order  of  the  level  of  acceptance. 

DEVICE  14B40 :  RADAR/MAD  OPERATOR  TRAINER.  The  14B40  was 
designed  to  train  sensor  operators  in  the  problems  and  pro¬ 
cedures  associated  with  non-acoustic  ASW  equipments,  specif¬ 
ically  radar  and  MAD  e*  ~:pment.  It  is  a  well  accepted 
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trainer  with  a  composite  criterion  score  of  +3.3.  As  indi¬ 
cated  by  Figure  22,  this  trainer  received  quite  high  average 
ratings  on  the  simulation  factors;  it  was  rated  above  average 
in  displa)  of  information,  equipment  performance,  control 
effects,  sensor  performance,  and  target  reactions.  However, 
the  ratings  on  these  factors  were  somewhat  below  the  judged 
importance  level. 

With  respect  to  training  and  situational  factors,  the 
14B40  generally  rated  about  the  average  of  all  trainers 
(Figure  23).  However,  it  was  rated  very  high  in  problem 
setup  and  above  average  in  training  effectiveness  in  com¬ 
parison  to  operational  equipment  and  to  other  similar  trainers. 

In  general,  the  instructors  felt  that  the  14B40  is  a 
very  effective  training  device  with  a  few  minor  improvements 
needed  such  as  "more  realistic  weather  effects  and  radar 
range  cutoff  points  due  to  altitude."  The  first  class  of 
students  trained  with  the  14B40  (it  is  a  new  trainer)  "im¬ 
pressed  the  squadron  people  with  the  amount  of  information 
they  carried  back  with  them." 

There  was  some  early  concern  on  the  part  of  maintenance 
people  about  the  reliability  of  the  trainer  that  may  be 
reflected  in  the  below  average  rating  on  reliability.  These 
fears  have  failed  to  materialize  according  to  the  personnel 
interviewed.  Part  of  the  problem  was  that  a  large  number  of 
changes  to  the  trainer  have  kept  maintenance  personnel  un¬ 
certain  concerning  whether  schematics,  manuals,  and  other 
technical  information  are  up-to-date.  Availability  of  parts 
has  been  a  problem,  especially  replacement  parts  for  GFE 
items.  There  reportedly  have  been  major  problems  with  the 
tactical  situation  display. 

"The  factory  school  for  this  trainer  was  struc¬ 
tured  for  computer  programming  with  no  real  help 
being  given  for  practical  maintenance  of  the 
systems.  Preliminary  manuals  are  still  in  use, 
having  not  been  replaced  by  the  final  manuals  by 
the  contractor." 

There  was  a  hint  of  slightly  greater  acceptance  of  the 
14B40  among  personnel  who  had  had  considerable  operational 
experience  with  radar  and  MAD  systems  (see  Table  15).  Ex¬ 
perienced  personnel  rated  the  trainer  noticeably  higher  on 
simulation  of  sensor  performance  than  neophyte  personnel 
but  rated  it  lower  on  reliability  and  level  of  training. 


t 
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TABLE  15.  AVERAGE  RATINGS  OF  THE  14B40 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


Low 

(N=5) 

Medium 

(N=6) 

Hi  gb 

L  IN -6) 

Average  Rating, 
Simulation  Factors 

9.5 

11.7 

■hh 

Average  Rating, 

Training  and 

Situational  Factors 

10.9 

9.6 

!  1  ,  1 

DEVICE  2F64A :  SH3D  HELICOPTER  CREW  TRAINER.  The  2F64A  is  a 
weapon  systems  trainer  used  to  train  SH3D  helicopter  crews 
in  ICS  procedures,  dipping  procedures,  crew  coordination, 
knobology,  and  practice  in  tracking.  It  received  a  somewhat 
above  average  composite  criterion  score  of  +2.0. 

The  2F64A  profile  on  simulation  factors  is  shown  in 
Figure  24.  It  is  apparent  that  the  trainer  approximated  the 
the  average  of  all  trainers  as  a  group  on  most  factors  al¬ 
though  it  was  rated  somewhat  above  average  in  display  simula¬ 
tion,  sensor  performance,  and  target  reactions.  However,  it 
failed  to  reach  the  judged  level  of  importance  in  these 
areas  and  also  shows  an  "importance  differential"  on  simula¬ 
tion  of  vehicle  performance  and  control  effects. 

/ 

With  respect  to  training  effectiveness  (Figu/e  25),  the 
trainer  was  also  rated  about  average  although  it  received  a 
superior  rating  on  immediacy  of  performance  evaluation  and 
somewhat  above  average  scores  on  value  of  trainer  time  and 
comparison  with  similar  trainers. 

During  the  interviews  complaints  were  recorded  concern¬ 
ing  the  realism  of  the  sonar  displays  and  limited  simulation 
of  the  effects  of  water  conditions  and  their  influence  on 
target  classification.  The  trainer  was  criticised  for  having 
no  capability  for  raising  or  lowering  the  dome,  and  poor  ICS 
communications . 

A  small  percentage  of  pilots  have  an  unfavorable  opinion 
of  the  device  because  they  feel  it  does  not  fly  like  a  heli¬ 
copter.  Administrative  and  instructor  personnel  reportedly 
try  to  combat  this  viewpoint  by  pointing  out  that  the  trainer 


TRAINING 


NAVTRADEVCEN  70-C-0276-1 


is  primarily  for  procedures  training.  The  slow  pace  of  up¬ 
dating  the  trainer  to  match  the  performance  characteristics 
and  other  features  of  the  actual  aircraft  have  created  some 
negative  opinions. 

"Pilot  trainees  feel  that  the  instructois  are 
familiar  with  trainee  reactions  and  the  average 
ability  of  students  but  they  are  not  able ‘to 
judge  the  ability  of  pilots  accurately  since 
they  are  not  pilots  themselves.  Pilots  feel 
that  the  trainer  is  excellent  for  system  NATOPS 
familiarization  and  cockpit  organization  and 
understanding.  However  they  do  n.?t  like  to  be 
evaluated  by  enlisted  personnel." 

"The  flight  characteristics  leave  a  lot  to  be 
desired.  There  is  too  much  instrument  lag  in  a 
few  instruments.  For  instance,  the  remote  mag¬ 
netic  indicator  and  the  vertical  gyro  indicator. 

There  is  no  rudder  stick  balance  with  the  needle 
ball  display.  Gyro  movement  is  not  realistic. 

The  trainer  will  not  simulate  hovering." 

The  sonar  equipment  is  reportedly  a  persistent  mainte¬ 
nance  problem. 

"Due  to  limited  use  in  the  past,  the  system  has 
not  been  kept  up  to  snuff.  Now  that  utilization 
has  increased,  increased  maintenance  is  required. 
Documentation  is  poor  and  we  must  proceed  by 
trial  and  error." 

"The  GFE  sonar  equipment  presents  maintenance 
problems.  Signal  input  levels  are  so  low  that 
infinitesimal  change  will  foul  the  whole  display." 

"The  reputation  of  the  trainer  has  improved  con¬ 
siderably  since  its  introduction  (at  Quonset 
Point).  The  primary  fault  of  a  digital  computer 
system  is  that  a  down  computer  means  a  down 
trainer.  Maintenance  people  did  not  know  enough 
about  the  digital  equipment  when  the  trainer  was 
new  and  sometimes  a  Chinese  fire  drill  would 
ensue  if  a  computer  quit.  The  trouble  in  some 
instances  was  external  to  the  computer.  The 
reputation  that  the  trainer  has  had  for  being 
bad  (unreliable)  is  not  necessarily  a  valid  one." 

As  indicated  in  Table  16,  there  were  no  apparent  differ¬ 
ences  in  overall  acceptance  of  the  2F64A  as  a  function  of 
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user  experience  in  the  SH3D.  However,  experienced  personnel 
rated  it  significantly  lower  on  training  software  and  com¬ 
pleteness  of  performance  evaluation. 

TABLE  16.  AVERAGE  RATINGS  OF  THE  2F64A 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


Low 

U=n) 

Medium 
(N=7 ) 

Hi  gn 
(N=8) 

Average  Rating, 
Simulation  Factors 

10.7 

9.7 

10.  1 

Average  Rating, 

Training  and 

Situational  Factors 

10.9 

10.0 

9.5 

DEVICE  2F66A--A  COMPARISON  OF  TWO  INSTALLATIONS  OF  THE  SAME 
TRAINER.  The  2F66A  was  developed  to  provide  pilot  and  crew 
training  for  the  S2E  ASW  aircraft.  Specific  emphasis  was 
placed  on  pilot  emergency  and  instrument  flight  procedures 
and  on  the  tactical  interaction  between  all  members  of  the 
crew.  Installations  at  both  San  Diego  (North  Island)  and 
Quonset  Point  were  studied  for  possible  differential  accep¬ 
tance  of  the  OFT  portion.  Ratings  were  obtained  on  the 
entire  trainer  at  San  Diego  but  only  the  OFT  portion  at 
Quonset  Point. 

As  indicated  in  Table  4,  neither  trainer  departed  mark¬ 
edly  from  the  overall  average  with  respect  to  the  composite 
criterion  of  acceptance.  However,  the  San  Diego  trainer, 
with  a  score  of  +1.1,  was  rated  about  three  units  higher  than 
the  Quonset  Point  trainer  with  a  score  of  -2.2.  This  was  due 
in  part  to  the  higher  utilization  at  San  Diego  but  also  to 
the  fact  that  a  greater  proportion  of  the  raters  at  Quonset 
Point  were  pilots.  It  was  evident  that  the  pilots  at  both 
locations  rated  the  2F66A  lower  than  did  the  tactical  crew 
members  because  of  what  they  felt  were  serious  deficiencies 
in  its  flying  characteristics. 

A  comparison  of  Figures  26  and  27  indicates  that  the 
Quonset  trainer  was  rated  particularly  low  on  vehicle  per¬ 
formance  and  effects  of  controls.  A  check  of  the  data 
showed  that  San  Diego  pilots  rated  that  trainer  as  similarly 
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deficient  in  those  areas.  However  the  higher  ratings  given 
the  San  Diego  trainer  by  tactics  personnel  obscured  the  sim¬ 
ilarities.  This  suggests  that  in  future  studies  the  opinions 
of  pilot  and  tactical  personnel  should  be  treated  separately 
in  analyzing  the  data.  Unfortunately,  in  the  present  study, 
it  was  not  possible  to  include  enough  personnel  in  each  cate¬ 
gory  to  get  reliable  results  for  different  sections  of  the 
trainer. 

The  other  differences  in  the  simulation  profiles  of  the 
two  trainers  can  similarly  be  explained.  The  San  Diego 
trainer  was  rated  somewhat  lower  on  information  displays  and 
sensor  performance.  This  probably  reflects  a  more  critical 
evaluation  of  those  characteristics  of  the  trainer  by  tac¬ 
tical  personnel. 

The  profiles  of  the  two  trainers  on  training  effective¬ 
ness  factors  are  very  similar  (Figures  28  and  29) .  Both 
received  average  ratings  in  most  respects  although  both  were 
rated  down  significantly  in  the  area  of  software.  The  reason 
for  the  unusually  high  rating  of  the  Quonset  trainer  on  level 
of  training  is  not  known.  It  may  again  reflect  the  inadequate 
sampling  of  tactical  personnel  at  the  Quonset  installation. 

Some  typical  comments  concerning  the  2F66A  at  Quonset 
Point  follow. 

"The  trainer  affords  good  ASW  tactics  practice, 
emergency  procedures  practice,  and  practice  in 
engine  management  during  unusual  circumstances." 

"Some  instructors  are  not  instructors  at  all. 

They  merely  turn  on  the  machine  and  push  buttons 
when  requested  to  do  so.  It  would  be  beneficial 
if  they  were  qualified  to  instruct.  They  know 
nothing  about  ASW,  flying,  etc." 

"Learning  to  fly  the  trainer  creates  some  possible 
negative  training  because  of  the  difference  in  the 
flying  techniques  that  must  be  used  in  mastering 
the^trainer  as  opposed  to  those  used  for  mastering 
an  aircraft.  Having  to  concentrate  on  the  handling 
ability  of  the  trainer  detracts  from  the  purpose  of 
training . " 

"I  believe  this  trainer  is  the  best  way  to  train 
a  VS  crew.  All  aspects  of  the  VS  mission  can  be 
accomplished  in  the  trainer  short  of  an  actual  cat 
shot  and  arrested  landing,  excluding  searchlight 
and  camera  pod  hops.  If  you  do  all  that  is  expected 
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of  you  in  the  trainer,  I  ielieve  you  could  fly 
an  actual  mission  as  far  ts  ASW  goes.  It  is  a 
valuable  training  aid  jto  '.ay  the  least." 

"Aircrewmen  would  like  to  ,*ee  spurious  contacts 
for  the  tactics  portion;  t.ere  are  few  environ¬ 
mental  qualities  available  i the  tactics  dis¬ 
plays  such  as  washover,  sea  return,  and  ambient 
noise . " 

The  following  are  selected  comments  about,  the  2F66A 
made  by  San  Diego  personnel. 

"The  trainer  is  very  effect ’.-we  if  properly  uti¬ 
lized.  It  simulates  real  conditions  so  well 
that  in  some  instances  frustrations  have  been 
induced  just  as  if  the  equipment  being  used  were 
real . " . 

"Effective  training  is  hindered  by  instructor 
capability.  When  they  don't  know  enough  about 
the  trainer  to  simulate  real  world  problems, 
the  training  suffers." 

"Instructor  techniques  seem  not  to  be  standardized. 
Newer  instructors  are  not  as  good  as  the  older 
ones.  A  lot  of  time  is  spent  in  stopping  the  prob¬ 
lems  and  resetting  them." 

"The  device  trains  very  well  in  'knouology,'  basic 
operator  functions,  instrument  flight  character¬ 
istics  for  the  pilot,  and  ASW  plotting..  It  is, 
however,  limited  in  the  scope  of  simulation." 

"The  trainer  is  being  used  to  train  skills  other 
than  those  originally  intended  by  the  designer. 

It  is  being  used  in  expanded  tactics.  The  trainer 
is  versatile  enough  so  that  the  individuals  who 
want  to  spend  time  with  it  can  learn  of  new  ways 
to  incorporate  it  into  the  development  of  their 
tactical  skills." 

"One  of  the  major  deficiencies  of  the  trainer  is 
that  the  equipment  does  not  keep  up  with  the 
actual  aircraft;  therefore  some  negative  training 
results.  Pilots  in  particular  are  sensitive  to 
these  differences." 

"The  trainer  is  too  perfect.  It  doesn't  provide 
for  the  gray  areas.  It  doesn't  teach  what  to 
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expect,  in  the  real  world.  The  signals  are  ideal; 
targets  are  easily  detected.*' 

"There  is  a  lack  of  realism  in  the  LOFAR  and  CODAR 
displays.  They  are  much  too  easy  to  deal  with. 

Some  of  the  systems  in  the  trainer  are  outdated 
such  as  the  JULIE  and  gram  analysis  capability 
with  the  JEZ  system.  Students  feel  that  tactics 
ard  conditions  are  changing  faster  than  it  is  pos- 
siw’e  to  update  the  training  equipment." 

"Schematics  are  very  poor  and  a  high  degree  of 
technical  background  is  necessary  to  use  the  ones 
that  are  provided.  The  technical  manual  is  no 
good  and  is  net  used.  Drawings  and  schematics 
are  lousy.  They  are  good  for  general  familiariza¬ 
tion  but  are  not  reliable  for  any  detailed  main¬ 
tenance  work." 

"Maintenance  personnel  had  to  set  up  their  own 
school  with  regard  to  learning  the  computer  and 
how  it  worked.  The  information  provided  by  NTDC 
was  much  too  skimpy." 

"Government  furnished  equipment  is  generally  poor 
and  extremely  hard  to  maintain.  An  example  on  i 

this  device  is  the  ASN-30  plotter.  The  Packard- 
Bell  250  computer  is  (also)  a  problem.  It  is  a 
general  purpose  computer  but  repair  parts  are 
scarce  and  very  slow  in  being  supplied." 

"As  an  OFT,  it  (2F56A)  is  of  value  only  to  the 
new  student  to  learn  systems  and  some  procedures. 

Visually  it  is  an  exact  duplication  of  the  air¬ 
craft,  but  control  pressure  and  responses  are  not 
at  all  like  the  aircraft." 

"As  a  WST,  it  is  of  value  for  learning  crew  co¬ 
ordination  and  procedures  and  for  solving  complex 
problems.  It  is  of  great  value  for  improving 
readiness;  if  all  training  was  to  be  obtained  in 
the  aircraft  the  time  and  expense  would  be 
prohibitive . " 

"I  would  also  comment  on  the  amount  of  down  time. 

It  can  be  infuriating  to  be  near  a  solution  of  a 
long  and  difficult  problem  and  have  the  computer 
dump . " 

t 

"During  a  training  session  when  a  whole  crew  was 
working  on  an  exercise  to  obtain  a  qual,  the 
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computer  dumps  the  problem  destroying  the  time 
limit  involved  and  the  procedure  necessary  for  a 
qual . " 

The  results  of  this  acceptance  analysis  of  the  2F66A 
trainers  appear  to  be  entirely  compatible  with  the  much  more 
detailed  evaluation  of  the  training  effectiveness  of  the  de¬ 
vice  recently  completed  by  the  Bunker-Ramo  Corporation  fl2). 

In  that  study,  Meistcr  et  al.  concluded  that  the  2F66A  was 
effective  for  training  all  crew  members  but  that  it  was  more 
effective  for  beginning  students  than  for  operational  and 
reserve  personnel.  The  trainer  received  particularly  low 
ratings  on  realism  of  the  control  forces,  and  realism  of 
information  displayed  to  the  sensor  operators,  and  only  aver¬ 
age  scores  on  "overall  effectiveness." 

While  the  instructors  considered  the  trainer  effective, 
problems  of  equipment  malfunctions  and  insufficient  fidelity 
"have  reduced  confidence  and  lowered  the  overall  accept¬ 
ability  of  the  device  as  an  integral  part  of  air  ASW  training. 
It  was  further  concluded  that  important  improvements  in 
training  for  all  S-2E  personnel  could  be  effected  through  im¬ 
proved  systematic  utilization  of  the  existing  capabilities* 
of  the  training  device. 

It  was  suggested  that  among  the  steps  that  could  te 
taken  to  improve  trainer  effectiveness  were  "(1)  more  sys¬ 
tematic  variation  of  the  nature  of  the  training  sessions; 

(2)  increased  trainer  usage;  (3)  more  systematic  utilization 
of  the  trainer  in  both  individual  position  and  team  training 
modes;  and  (4)  improved  trainer  maintenance,  enhanced  fidel¬ 
ity,  and  increased  trainer  hardware  capability.' 

Table  17  suggests  that  these  trainers  are  somewhat  less 
accepted  by  the  more  experienced  personnel.  Both  trainers 
were  rated  particularly  low  by  experienced  personnel  on 
software.  In  addition,  the  Quonset  trainer  was  rated  very 
low  on  sensor  simulation  and  the  San  Diego  trainer  was  rated 
particularly  low  on  control  effects. 
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TABLE  17.  AVERAGE  RATINGS  OF  THE  2F66A 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 

Experience  Level 


2F66A 

(Quonset  Point) 

low 
( N— 2 ) 

Medi urn 
(N  =  4) 

Hi  gh 

.  .  <N=4) 

Average  Rating, 
Simulation  Factors 

9.2 

9.5 

9.6 

Average  Rating, 

Training  and 

Situational  Factors 

12.6 

9.6 

6.8 

2F66A 

(San  Dieqo) 

Low 

(N=8) 

Medi urn 
(N- 8) 

High 
(N  =  8) 

Average  Rating, 
Simulation  Factors 

1  1  .0 

10.5 

9.6 

Average  Rating, 

Training  and 

Situational  Factors 

10.4 

10.  1 

8 . 4 

I 


DEVICE  14A6A:  ASW  COORDINATED  TACTICS  TRAINER.  The  objective 
of  the  14A6A  was  to  provide  training  in  coordinated  ASW 
operations  varying  from  a  relatively  simple  situation  involv¬ 
ing  a  few  ASW  units,  to  complex  exercises  involving  an  entire 
ASW  task  group  (up  to  2S0  key  people) .  The  trainer  was  to  be 
utilized  to  conduct  demonstrations,  provide  for  coordinated 
training,  and  evaluate  tactics.  It  was  designed  to  illus¬ 
trate:  (1)  the  capabilities  and  limitations  of  surface  ships, 

submarines,  aircraft,  helicopters,  and  ASW  task  groups:  (2) 
the  utilization  of  search,  localization  and  attack  equipments 
and  weapons;  (3)  the  ccntrol  of  multi-unit  forces  in  conduct¬ 
ing  search,  localization  and  attack;  (4)  communication, 
plotting  and  recording  facilities  and  procedures  for  the 
rapid  exchange  and  analysis  of  necessary  information;  (S)  com¬ 
mand  relationships  and  responsibilities;  (6)  realistic  enemy 
submarine  threats;  and  (7)  offensive*  and  defensive  missions. 

The  trainer  was  to  provide  training  for  decision-making 
personnel  in  coordinated,  inter -type,  ASW  tactical  situations. 
The  paiticipants  include  aviation,  surface,  and  submarine 
personnel  from  enlisted  through  flag  rank.  In  genera],  it  was 
expected  that  the  participants  would  be  "moderately  to  ex¬ 
pertly  skilled  in  their  particular  individual  t>pe  functions; 
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however,  they  may  be  found  to  require  practice,  to  varying 
degrees,  in  applying  coordinated  inter-type  ASW  tactics." 

In  terms  of  the  composite  acceptance  criterion  (Table 
4),  the  14A6A  trainer  fell  almost  in  the  middle  of  the 
distribution  on  the  acceptance  scale  with  a  score  of  +0.6. 

Its  level  of  acceptance  can  clearly  be  described  as  "modest." 
The  reasons  for  this  are  probably  most  evident  from  the 
profile  on  simulation  factors  (Figure  30).  It  is  clear  that 
the  14A6A  was  rated  below  the  average  of  all  trainers  on  all 
simulation  factors.  It  was  rated  particularly  low  on  simula¬ 
tion  of  controls,  displays,  sensor  performance,  target 
reactions,  a^d  weapons  firing  and  control.  It  is  interesting 
to  note,  fro -i  the  ratings  of  importance  on  these  factors, 
that  the  participants  did  not  regard  high  fidelity  simulation 
of  sensor  performance,  weapons  and  displays  as  important  for 
this  trainer  as  it  was  for  individual  ASW  weapons  systems 
trainers  (for  example,  the  14A2).  Nevertheless,  the  importance 
ratings  were  considerably  higher  than  the  values  assigned  to 
the  14A6A  in  many  important  respects. 

In  the  contrast  to  the  ratings  on  simulation  factors, 
the  14A6A  was  rated  considerably  more  favorably  with  respect 
to  general  training  effectiveness  factors  (see  Figure  31). 
Dissatisfaction  is  evident,  however,  with  respect  to  the 
completeness  and  immediacy  of  performance  evaluation.  It 
received  above  average  ratings  in  reliability  and  level  of 
training.  With  respect  to  most  other  characteristics,  it 
did  not  depart  in  any  pronounced  way  from  the  average  of  all 
trainers . 

The  failure  of  the  14A6A  to  meet  the  levels  judged  as 
important  on  various  simulation  factors  perhaps  reflects  a 
frequently  mentioned  criticism  of  the  trainer,  namely,  that 
considerable  time  and  effort  must  be  expended  learning  to 
operate  the  trainer  per  se.  The  equipment  in  the  trainer  is 
"so  unlike  operational  equipment  that  it  is  necessary  to 
spend  too  much  time"  learning  its  peculiarities. 

A  possibly  significant  deficiency  is  that  since  an  entire 
ship's  team  occupies  a  single  small  space,  many  problems  in 
internal  communications  can  be  avoided  by  direct  face-to-face 
conversation,  or  by  looking  at  a  plot  or  display  not  normally 
available.  In  addition,  the  display  presents  unambiguous 
information  which  makes  the  problems  of  communication  un¬ 
realistic.  The  highest  rating  received  by  the  14A6A  was  in 
the  area  of  communications  procedures.  This  supports  the 
contention  of  many  that  the  14A6A  is  primarily  a  communications 
procedures  treiner.  Nevertheless,  internal  communications 
are  poorly  simulated. 
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With  respect  to  controls,  the  14A6A  equipment  is  very 
unlike  operational  equipment.  The  raters  accept  this  in 
principle,  but  nevertheless  think  it  should  be  more  realistic 
than  it  is. 

The  sensors  in  the  14A6A  are  very  unlike  the  real  sen¬ 
sors  and  they,  again,  provide  unambiguous  information.  With 
respect  to  target  reactions,  the  trainees  "know  that  the 
instructors  are  playing  games."  One  commanding  officer  com¬ 
mented  that  he  very  seldom  heard  any  discussion  of  how  foreign 
submarines  are  known  or  suspected  to  act  in  the  various  tac¬ 
tical  situations  presented  in  the  trainer. 

With  respect  to  weapon  control  and  firing,  it  is  em¬ 
phasized  that  the  14A6A  was  designed  to  carry  the  exercise 
up  to  the  point  of  firing  only.  Evidently  the  assumption 
was  that  the  firing  phase  of  training  would  be  taken  care  of 
in  other  trainers.  The  very  low  rating  on  this  factor  clearly 
indicates  the  opinion  of  many  user  personnel  that  this  is  not 
good  enough.  Several  trainees  and  a  few  instructors  com¬ 
mented  that  the  ideal  trainer  would  be  one  with  the  general 
capability  of  the  14A'6A  but  composed  of  several  14A2  mock-ups 
with  their  much  more  detailed  capabilities. 

The  comparatively  low  ratings  on  performance  evaluation 
probably  reflect  the  fact  that  there  is  no  provision  for  im¬ 
mediate  feedback  at  critical  points  in  the  exercise.  Exer¬ 
cises  typically  last  a  full  half  day  with  problem  reconstruc¬ 
tion  on  a  large  screen  at  the  end.  During  large  scale 
exercises,  enlisted  personnel  often  take  notes  at  individual 
stations;  however,  there  is  little  in  the  way  of  critiques 
of  individual  ship’s  actions.  What  there  is  tends  to  take 
the  form  of  "you  should  have  done  thus  and  so"  rather  than 
a  diagnostic  determination  of  why  a  particular  action  was  or 
was  not  taken  by  a  given  unit.  <  The  performance  of  individual 
operators  is  not  critiqued,  of  course,  because  they  are  not 
utilizing  actual  operational  gear.  An  exception  to  these 
observations  is  the  evaluation  of  the  overall  communication 
plan  that  is  established  for  the  purposes  of  the  exercise. 

This  is  evaluated  rather  thoroughly. 

The  reliability  of  the  trainer  on  the  whole  was  rated 
very  high.  However,  it  is  not  unusual  to  have  crews  shift 
from  one  mock-up  to  another  because  of  an  equipment  casualty. 
Since  the  trainer  is  rarely  used  to  full  capacity,  this  does 
not  result  in  a  significant  loss  of  training  time.  The  tech¬ 
nicians  complain,  however,  that  the  14A6A  was  built  using  a 
line  of  parts  left  over  from  other  projects.  They  reportedly 
often  discover  that  the  part  they  require  is  no  longer  in  the 
Navy  supply  system  or,  worse,  was  built  by  a  company  no  longer 
in  business . 
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It  seems  likely  that  the  ratings  of  the  14A6A  would  have 
been  much  higher  in  the  absence  of  some  of  the  simulation 
shortcomings  mentioned  earlier.  However,  this  is  little 
doubt  that  the  trainer  helps  to  recognize  and  correct  many 
procedural  problems  before  the  units  go  to  sea,  particularly 
when  used  prior  to  a  major  fleet  exercise.  "Everyone  would 
prefer  more  at  sea  training,  but  since  that  is  not  possible, 
the  A6  is  a  better-than-nothing  substitute." 

There  is  evidence  in  the  data  (Table  18)  that  the  sim¬ 
ulation  deficiencies  are  more  strongly  felt  by  personnel 
with  the  greater  experience.  This  was  true  particularly 
with  respect  to  communication  problems,  displayed  informa¬ 
tion,  and  weapon  firing  effects. 

TABLE  18.  AVERAGE  RATINGS  CF  THE  14A6A 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


DEVICE  2821 :  T-28  INSTRUMENT  TRAINER.  The  2B21  is  an  in¬ 

strument  flight  trainer  with  multiple  cockpit  installations. 

It  is  designed  for  basic  training  in  the  (1)  operation  and 
use  of  individual  instruments;  (2)  development  of  an  in¬ 
strument-scan  pattern;  (3)  coordination  of  controls  and 
instruments  in  basic  flight  patterns;  (4)  use  of  radio  navi¬ 
gation  facilities,  i.e.. ,  VOR/TACAN,  and  ADF  and  ILS;  (S) 
CCA-landing  practice;  (6)  voice  procedures,  where  applicable 
in  the  above  areas  of  training;  and  (7)  operational  flight 
training  of  the  T-28B  aircraft. 

The  trainer  was  to  simulate  "normal  aircraft  operation 
with  respect  to  flight  and  engine  performance,  flying 
qualities,  aircraft  systems  operations,  radio  communication 
and  navigation  systems  operation,  and  flight  path."  The 
simulation  was  to  result  in  appropriate  response  on  the  part 
of  the  trainer's  indicators,  control  reactions,  and  the  flight 
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path  recorder.  The  trainees  utilizing  the  trainer  were 
presumed  to  have  no  prior  experience  in  instrument  flight 
and  navigation,  and  only  limited  flight  instruction. 

The  composite  criterion  score  for  the  2B21  was  +1.4, 
just  slightly  above  the  average  of  all  trainers.  It  is 
evident  that  the  trainer  is  accepted  but  not  without  some 
reservation.  The  reasons  may  be  reflected  in  the  profile 
on  simulation  factors  (Figure  32).  The  trainer  was  rated 
well  below  average  in  simulation  of  vehicle  performance  and 
control  effects.  In  relation  to  judged  importance,  these 
deficiencies  are  particularly  severe.  In  contrast,  the 
trainer  was  rated  as  distinctly  above  average  on  information 
displays  and  sensor  performance. 

The  2B21  was  rated  above  average  on  several  important 
training  effectiveness  factors.  These  include  problem  set¬ 
up,  software,  performance  evaluation,  reliability,  and 
value  of  time  in  the  trainer.  However,  the  trainer  was 
rated  below  average  on  level  of  training,  comprehensiveness 
of  training,  and  on  use  of  the  trainer  versus  operational 
equipment.  It  was  also  rated  somewhat  below  average  in  com¬ 
parison  with  other  trainers  of  this  type.  (See  Figure  33.) 

Some  possible  reasons  for  the  mixed  attitudes  of  accep¬ 
tance  and  rejection  are  reflected  in  the  following  comments. 

"The  trainer  does  not  fly  like  the  real ' ai rcraft . 

It's  a  poor  basic  air  trainer  but  a  very  good 
scan  procedures,  basic  instrument,  and  radio  work 
instrument  trainer.  Without  it,  students*  progress 
would  be  slow." 

"Some  instructors  feel  that  the  device  hurts  scan 
and  some  basic  procedures  and  also  basic  air  work 
for  trim,  rudders,  etc.  Making  the  trainer  re¬ 
semble  the  T28  aircraft  may  have  been  a  mistake. 
Students  expect  the  trainer  to  respond  like  the 
aircraft  and  it  was  not  designed  to  do  that. 

Lack  of  realism  in  the  control  responses  of  the 
trainer  is  a  continuing  problem  for  the  instructors." 

"Rudder  pedals  in  the  trainer  are  oversensitive. 
Students  transmit  information  to  each  other  to  the 
effect  that  they  should  fly  the  trainer  with  their 
feet  on  the  floor.  Some  negative  training  occurs 
because  of  this  condition.  They  forget  or  have 
negative  carry-over  and  try  to  fly  the  aircraft  in 
the  same  way." 
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Figure  32.  Profile  of  Simulation  Scores  for  Device  2B21 
Trainer,  Pensacola. 
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"The  turn  and  bank  indicator  does  not  react  to 
the  application  of  power.  Control  sticks  are 
not  smooth,  seem  to  bind,  and  cause  the  trainee 
to  over-control  the  trainer." 

"No  status  accrues  to  the  student  as  a  result  of 
flying  the  trainer.  They  are  still  playing  the 
"hours  game"  for  status  and  therefore  time  in  the 
trainer  only  detracts  from  the  time  they  have 
available  to  accumulate  hours  in  the  air." 

"The  more  intelligent  students  appreciate  the  in¬ 
tent  of  the  trainer  and  benefit  more  from  the 
training  time.  They  recognize  how  the  trainer 
satisfies  training  needs  in  the  area  of  instrument 
scan  procedures,  and  voice  reports,  but  they 
recognize  also  that  it  is  certainly  no  good  for 
basic  air  work.  Flying  the  "Link,"  as  they  refer 
to  it,  is  not  at  all  like  flying  the  aircraft. 

Some  students  feel  that  flying  a  Link  can  be 
detrimental  to  flying  in  the  aircraft  because  of 
great  differences  in  control  pressure  and  trim." 

It  is  noteworthy  that  student  responses  to  the  question¬ 
naires  in  this  study  were  very  similar.  They  expressed 
disappointment  that  they  could  not  increase  their  airmanship 
proficiency  in  the  trainer  and  were  concerned  by  the  un¬ 
reliability  of  the  internal  communications  system  between 
instructor  and  student.  The  trim  and  rudder  problems  were 
very  distracting  to  them.  Since  this  was  their  first  ex¬ 
posure  to  air  trainers,  it  is  possible  that  the  deficiencies 
noted  will  have  a  lasting  effect  on  the  opinion  of  some 
students  with  regard  to  air  trainers  in  general. 

There  were  no  large  differences  in  acceptance  of  the 
2B21  as  a  function  of  level  of  experience  (see  Table  19)., 
However  it  is  interesting  that  the  more  experienced  personnel 
rated  the  trainer  higher  on  level  of  training  and  value  of 
time  in  the  trainer  than  did  those  with  low  experience.  On 
the  other  hand,  they  rated  it  lower  in  comparison  with  similar 
trainers . 
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TABLE  19.  AVERAGE  RATINGS  OF  THE  2B21 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


~  Low 
(N=1 8) 

Medi um 
(N  =  3) 

r — RTgfR - 

(N=6 ) 

Average  Rating, 
Simulation  Factors 

9.3 

9.9 

9.6 

Average  Rating, 

Training  and 

Situational  Factors 

9.7 

9.6 

10.8 

DEVICE  2F6S :  E-2B  WEAPONS  SYSTEMS  TRAINER.  The  2F65  was 
designed  to  simulate  the  tactical  mission  of  the  E-2B 
weapons  system.  The  E-2B  is  an  all-weather  airborne  early 
warning  and  control  airplane.  It  has  a  tactical  crew  con¬ 
sisting  of  an  ECM  operator  (copilot),  CIC  officer,  air  con¬ 
trol  officer,  and  radar  officer  who  are  responsible  for  the 
operation  and  control  of  all  avionics  equipment.  The  co¬ 
pilot  doubles  as  ECM  operator  when  the  aircraft  is  on  station 
and  provides  ECM  inputs  to  the  avionics  system  computers. 

The  tactics  system  of  the  E-2B  performs  the  following 
functions:  detection,  acquisition,  automatic  tracking  and 

identification  of  targets,  report  of  target  positions,  and 
control  of  interceptors. 

The  device  was  intended  for  the  use  of  carrier  based  AEW 
squadrons,  to  train  fleet  pilots  and  AEW/CIC  crews  under  the 
direction  of  qualified  instructors.  The  device  was  to  pro¬ 
vide  training  in  all  of  the  necessary  cockpit  techniques, 
flight  planning,  emergency  procedures,  instrument  flight  pro¬ 
cedures,  and  tactical  procedures  associated  with  the  weapons 
system. 

On  the  composite  criterion  (Table  4),  the  2F65  received 
a  somewhat  below  average  overall  rating  (-2.2).  In  the  pro¬ 
file  on  simulation  factors  (Figure  34),  it  is  evident  that 
deficiencies  were  felt  in  the  areas  of  vehicle  and  equipment 
performance,  control  effects,  communication  procedures,  and 
information  displays.  In  addition,  the  trainer  was  rated 
distinctly  below  the  importance  ratings  with  respect  to  sen¬ 
sor  performance,  target  reactions,  and  weapons  control. 
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With  respect  to  training  effectiveness  factors,  the 
device  fared  much  better  (see  Figure  35).  The  trainer  was 
generally  ratod  average  or  better  in  al?  respects  with  the 
exception  of  software.  Despite  any  of  the  felt  deficiencies 
already  mentioned,  the  value  of  time  spent  in  the  trainer 
was  rated  as  above  average. 

It  was  clear  to  the  study  team  that  the  2F65  had  two 
distinct  levels  of  acceptance.  The  OFT  portion  was  generally 
regarded  not  acceptable  because  of  the  problems  associated 
with  simulated  flight,  particularly  aircraft  control  and 
response.  Evidently  the  analogue  simulation  of  these  func¬ 
tions  has  proven  very  unsatisfactory,  and  changes  in  aircraft 
performance  parameters  are  not  easily  incorporated  in  the 
trainer.  Reportedly,  "many  pilots  lose  sight  of  the  effec¬ 
tiveness  of  the  trainer  for  emergency  and  procedural  training 
because  of  these  deficiencies." 

The  tactics  section  of  the  trainer  has  a  much  higher 
level  of  acceptance  than  the  OFT.  This  appears  to  be  because 
it  can  be  used  to  develop  problem  solving  ability  and  perfor¬ 
mance  skills  by  NFO's.  Acceptance  is  not  unqualified,  how¬ 
ever.  NFO's  feel  they  would  derive  more  from  the  training 
sessions  if  "greater  realism  at  different  levels  of  complexity" 
could  be  incorporated  into  the  video  displays. 

Squadron  personnel  r, re  not  always  as  clear  as  they  might 
be  concerning  the  potential  value  of  the  trainer  and  the 
training  personnel  are  not  considered  fully  responsive  to 
their  training  needs.  No  one  seems  completely  happy  with  the 
trainer.  Major  updates  have  been  made  and  are  ongoing  but 
the  consensus  seems  to  be  that  the  effort  is  futile.  Because 
of  original  inadequacies,  it  is  felt  that  little  can  be 
accomplished  to  make  the  2F65  into  an  entirely  acceptable 
training  aid.  Early  unrewarding  experiences  seem  to  have 
carried  over  and  projected  into  future  expectations. 

Some  verbatim  responses  from  the  questionnaires  follow. 

"This  trainer  satisfies  needs  fer  training  in  voice 
reporting  and  manual  tracking;  exposes  students  to 
data  link  operations;  acts  as  an  interface  between 
the  classroom  and  the  real  world;  and  provides  a 
base  for  performing  and  experimenting  with  tactics,1’ 

"The  trainer  does  not  realistically  simulate  video 
modes,  no  side  lobes,  etc.  Raa„r  interpretation 
is  lacking  and  communications  net  background  noise 
is  also  lacking." 
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"More  tactics  problems  are  needed,  such  as  carrier 
controlled  approaches,  strike  controlled  fleet 
exercises 


"R 


and  war  games.11 
Jar  and  IFF  simulation  are  not  worthwhile." 


"It  is  felt  that  the  trainer  could  benefit  from 
Navy  man  as  the  primary  coordinator  for  all 
activities  related  to  the  trainer... a  managing 
director  who  can  better  relate  the  use  of  the 
trainer  to  the  operational  wo^ld,  a  real  Navy 
NFO/E23  type  with  rufficient  expertise  to  ramrod 
this  show  and  keep  on  top--this  would  be  his 
primary  duty.'1 

"Instructors  at  some  activities  do  not  thoroughly 
know  the  operations  and  limitations  of  airborne 
equipment.  Some  instructors  have  not  been  checked 
out  in  operation  of  the  trainer  when  they  are 
assigned  to  it.  Occasionally  both  of  these  con¬ 
ditions  apply  to  the  same  instructor." 

"The  operators  and  other  people*  associated  with 
the  trainer  have  no  general  flight  experience  in 
the  type  of  aircraft  they  are  trying  to  simulate. 

It  would  be  invaluable  to  have  these  people  ex¬ 
perience  familiarization  flights  in  the  aircraft 
itself." 

"More  rapid,  frequent  and  direct  contact  is  re¬ 
quired  with  project  personnel  both  at  NTDC  and 
the  Naval  Air  Systems  Command,  These  support 
personnel  have  the  most  current  information  on 
trainer  status,  use  and  need  and  are  at  the  end 
of  the  chain  of  decision.  They  are  the  last  to 
know  concerning  the  many  items  affecting  the 
ability  of  the  trainer  to  meet  its  mission.  There 
is  no  wish  on  the  part  of  the  lower  people  to 
usurp  power;  they  would  just  like  to  be  provided 
with  a  method  for  making  valid  inputs  for  trainer 
changes  and  improvements  and  they  would  like  to 
be  informed  of  what  the  training  requirements  are 
going  to  be." 

"Problems  occur  in  troubleshooting  because  of  the 
way  the  schematics  and  other  parts  of  the  manuals 
are  laid  out  and  these  cause  considerable  confusion 
and  wasted  tine.  The  trainer  has  undergone  ex¬ 
tensive  updating  which  has  required  a  lot  of  revision 
of  the  technical  manuals.  Schematics  and  drawings 
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are  for  the  most  part  insufficient.  Some  are  too 
brief  or  vague  to  be  of  much  help;  some  are  so 
full  of  errors  they  are  useless.  Technical  man¬ 
uals  appear  to  be  written  by  people  who  take  an 
awful  lot  for  granted,  for  example,  engineers. 

Parts  of  some  systems  are  scattered  over  many 
sections  of  the  manuals. 

"Training  manuals  are  not  used  regularly  because 
the  individuals  are  familiar  enough  with  the 
equipment.  Write-ups  in  the  manual  are  weak  and 
don't  necessarily  relate  to  the  training  equip¬ 
ment.  Scuttlebutt  has  it  that  the  documentation 
was  subcontracted  and  developed  from  engineering 
notes  . " 

"There  needs  to  be  a  more  responsive  method  for 
getting  aircraft  changes  incorporated  into  the 
trainer  quickly.  The  decision  to  include  air¬ 
craft  changes  (in  the  trainer)  are  too  far  re¬ 
moved  (in  time)  from  operational  (changes). 

Negative  training  occurs  because  of  this," 

Some  observations  from  trainees: 

"The  trainer  is  good  for  cockpit  familiarization, 
normal  pre-starts,  starts,  takeoff  and  landing 
procedures,  and  eye  scan  development.  The  students 
are  generally  left  on  their  own.  Some  come  to  the 
trainer  very  well  prepared;  others  come  with  a 
lackadaisical  attitude,  sometimes  fostered  by  pre¬ 
vious  troubles  in  the  trainer.  The  students  feel 
that  the  trainer  doesn't  simulate  conditions  well 
enough  to  do  much  training.  This  applies  to  the 
OFT  portion.  The  opinion  is  that  the  trainer  bears 
no  resemblance  to  the  E2  aircraft.  It  is  good  for 
emergency  procedures  within  limitations  and  it 
makes  a  good  cockpit  familiarization  trainer.  But 
it  doesn't  fly  too  well  and  the  navigational  aids 
don't  work.  Very  frequently  it  fails  during  flight." 

"Many  instruments  are  too  imprecise  to  give  accurate 
simulation.  Many  programs  were  not  included  or  were 
erroneously  installed  and  reprogramming  is  difficult 
and  expensive.  Flight  curves,  etc.,  would  need  re¬ 
scaling  to  make  them  match  the  performance  of  the 
aircraft.  The  original  design  information  was  in¬ 
accurate  and  since  these  features  arc  hard-wired 
into  the  trainer,  there  is  little  opportunity  for 
change . " 
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"It  is  felt  that  direct  liaison  between  squadron 
NATOPS  people  and  NTDC  is  necessar> .  At  present 
many  inputs  (suggestions  for  change)  are  received 
at  widely  separated  periods  of  time.  What  is 
needed  is  continuous  liaison;  also,  most  necessary 
changes  are  beyond  the  capability  of  NTDC  repre¬ 
sentatives.  Much  effort  was  expended  by  NTDC  to 
fix  the  OFT  portion  of  the  trainer.  But  most  of 
what  was  initiated  was  inconsequential  insofar  as 
training  was  concerned.  The  present  trainer  is 
beyond  the  scope  of  changes  necessary  to  bring  it 
up  to  date." 

One  respondent  had  had  experience  previously  with  a  com¬ 
mercial  airlines  simulator.  He  felt  the  difference  was 
startling  between  the  commercial  simulator  and  the  2F65  and 
that  the  Navy  could  benefit  by  incorporating  some  of  the 
features  used  in  commercial  simulators.  Quite  a  few  trainees 
were  aware  of  what  the  airlines  were  doing  with  simulators. 
They  felt  that  trainers  can  be  made  to  do  the  job  but  that 
this  one  was  not.  Their  question  was,  "Why?" 

The  more  experienced  personnel  were  definitely  less 
accepting  of  the  2F65  than  neophyte  personnel  (see  Table  20), 
This  was  particularly  true  in  the  areas  of  vehicle  perfor¬ 
mance,  communication  problems,  displayed  information,  problem 
setup,  training  software,  completeness  of  performance  evalua¬ 
tion,  level  of  training,  and  value  of  trainer  us.  operational 
experience . 


TABLE  20.  AVERAGE  RATINGS  OF  THE  2F65 
BY  LEVEL  OF  EXPERIENCE  OF  USERS 


Experience  Level 


Low 

(N=3) 

Medi urn 
(N  =  5) 

Hi  gh 
(N-6 ) 

Average  Rating, 
Simulation  Factors 

10.9 

8.5 

8.4 

Average  Rating, 

Training  and 

Situational  Factors 

11,8 

10.4 

7.6 
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A  POST  HOC  APPRAISAL  OF  THE  RATING  SCALES 
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One  of  the  objectives  of  this  study  was  to  develop  a 
methodology  which  could  be  used  in  the  future  by  NTDC  to 
assess  the  acceptance  of  training  devices.  It  is  felt  that 
the  acceptance  profiles  presented  in  the  preceding  section 
are  clearly  useful  and  diagnostic.  Nevertheless,  it  would 
be  surprising  if  the  rating  scales  from  which  they  were 
derived  could  not  be  improved.  Those  familiar  with  the  use 
of  rating  scales,  particularly  by  personnel  who  are  not 
trained  in  their  use,  will  be  well  aware  that  the  judges  do 
not  necessarily  discriminate  effectively  on  all  of  the  char¬ 
acteristics  (i.e.,  trainer  features)  described. 

Undoubtedly,  many  of  the  characteristics  are  correlated, 
i.e.,  they  are  not  logically  independent  of  one  another.  It 
is  also  possible,  however,  that  the  ratings  of  training  de¬ 
vices  could  suffer  from  a  sort  of  generalized  "halo"  effect, 
a  commonly  observed  phenomenon  when  supervisors  rate  their 
personnel  on  performance  effectiveness.  The  "halo"  effect 
occurs  when  one  salient  feature  of  a  device  (or  individual) 
produces  a  generalized  reaction,  either  negative  or  positive, 
such  that  the  rater's  judgments  of  all  other  characteristics 
are  biased  in  a  similar  direction  by  this  feature.  In  the 
limiting  case,  the  judgment  of  a  single  characteristic 
"colors"  the  judgment  of  all  others,  and  only  a  single  dimen¬ 
sion  is  required  to  extract  all  of  the  information  obtainable 
from  the  rating  scores. 

In  the  case  of  training  devices  it  would  be  hoped  that 
the  users  would  judge  each  characteristic  relatively  inde¬ 
pendently  of  the  others.  This  would  be  reflected  by  com¬ 
paratively  low  correlations  among  the  ratings  assigned  to 
different  features  of  the  trainer.  On  the  other  hand,  if 
there  were  a  strong  "halo"  effect,  the  correlations  would 
be  very  high. 

Throughout  this  report  we  nave  portrayed  the  results  as 
if  the  simulation  and  training  effectiveness  factors  were 
truly  independent  in  the  minds  of  the  participants  even 
though  we  knew  that  "halo"  effects  might  have  effectively 
reduced  the  number  of  dimensions  on  which  the  trainers  were 
being  judged.  In  the  interest  of  improving  the  methodology, 
and  the  definition  of  the  composite  criterion  of  acceptance, 
a  detailed  statistical  analysis  was  therefore  performed  in 
an  attempt  to  better  identify  the  nature  and  number  of 
underlying  dimensions  on  which  the  trainers  were  actually 
judged. 
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Pearson  product-moment  correlations  were  computed  for 
the  scores  of  all  trainers  on  the  22  simulation  and  training 
characteristics.  The  resulting  correlation  matrix  was  sub¬ 
jected  to  factor  analysis,  a  technique  for  identifying  the 
smallest  number  of  common  factors  (dimensions)  that  account 
for  all  of  the  common  variance  in  the  matrix.  Varimax  rota¬ 
tion  to  oblique  simple  structure  was  performed  in  an  effort 
to  maximize  the  interpretabi  lity  of  the  results. 

The  oblique  primary  factor  loadings  resulting  from  this 
procedure  are  shown  in  Table  21.  For  convenience,  10  factors 
were  extracted  which  the  computer  program  clearly  indicated 
was  more  than  enough  to  account  for  the  common  variance. 

Not  all  of  the  factors  met  conventional  statistical  criteria 
but  it  was  evident  that  7  of  the  10  were  clearly  interpret¬ 
able  on  logical  grounds.  Each  of  these  factors  or  under¬ 
lying  dimensions  is  briefly  described  below. 

Factor  1.  Recognized  Value.  The  rating  scale 
variables  that  loaded  heavily  on  this  factor 


were  as  follows  : 

Importance  of 

.training  opportunities  provided  .76 
Value  of  repeated  use  .69 
Level  of  training  .68 
Use  of  trainer 

versus  operational  equipment  .59 
Comprehensiveness  of  training  .49 
Value  of  time  in  the  trainer  .38 


It  seems  evident  that  this  factor  is  a  rather  direct 
reflection  of  the  felt  value  of  the  training  device  in  the 
overall  scheme  of  training.  It  is  a  clear  reflection  of 
the  fact  that  training  devices  are  recognized  as  fulfilling 
a  highly  important  need  in  addition  to  the  opportunities  for 
practice  with  operational  equipment.  This  is  undoubtedly  a 
basic  underlying  consideration  in  the  acceptance  of  any 
training  device. 
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Factor  2.  Tactical  Realism.  The  features  of  a 
trainer  that  loaded  most  heavily  on  this  factor 
had  to  dc  with  simulated  sensor  performance, 
representation  of  targets,  and  weapon  performance. 
All  of  the  rating  scale  variables  comprising  this 
factor  relate  to  the  adequacy  with  which  the  tac¬ 
tical  problem  is  presented. 


Simulation 

of 

sensor 

performance 

.76 

Simulation 

of 

target 

evasion 

.71 

Simulation 

of 

weapon 

firing 

.58 

Simulation 

of 

displays 

.48 

Factor  3.  Provision  for  Evaluation.  This  factor 
was  defined  by  only  two  rating  scale  character¬ 
istics  : 

Immediacy  of  performance  evaluation  .80 

Completeness  of 

performance  evaluation  .77 

Although  the  definition  of  a  "common"  factor  technically 
requires  high  loadings  by  more  than  two  variables,  these  two 
characteristics  clearly  logically  identify  a  basic  dimension 
of  trainers  that  is  wel]  defined  in  the  minds  of  evaluators 
of  training  devices.  No  other  characteristic  of  the  trainer 
showed  even  a  modest  loading  on  this  factor. 

Factor  4,  Performance  "Feel".  This  factor  was 
also  a  "doublet"  with  heavy  loadings  on  two  char¬ 
acteristics  that  are  logically  related  in  training 
devices.  These  were: 

Simulation  of  controls  .75 

Simulation  of  performance  ,67 

There  were  no  other  significant  loadings  on  this  factor. 
It  is  clearly  a  factor  that  is  of  major  concern  tc  users  of 
operational  flight  trainers  and  other  devices,  such  as  sub¬ 
marine  diving  trainers,  which  are  characterized  by  extensive 
feedback  to  the  operator  through  the  controls  and  apparent 
motions  of  the  vehicle. 
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Factor  5.  Operational  Readiness .  The  trainer 
characteristics  loading  on  this  factor  included 
both  maintenance  and  operational  considerations. 
Those  with  the  heaviest  loadings  were  as  follows: 


Reliability  and  maintainability 

.69 

Problem  setup 

.57 

Training  software 

.49 

This  factor  appears  to  reflect  the  "readiness"  of  the 
device  to  meet  training  needs  from  both  a  reliability  and 
training  convenience  viewpoint.  It  undoubtedly  reflects  an 
important  dimension  in  acceptance,  both  by  students  and  by 
the  personnel  who  must  keep  devices  operational. 

Factor  6.  Communication  Provisions .  The  only 
rating  scale  variables  loading  heavily  on  this 
factor  were  the  two  items  describing  provisions 
for  communications  in  the  trainer.  The  load¬ 
ings  were  as  follows: 

Simulation  of 
communication  procedures 

.66 

Simulation  of 
communication  problems 

.58 

Factor  7.  Environmental  Simulation. 
another  example  of  a  doublet  that  is 
well  defined.  In  this  case  the  only 
scale  variables  with  heavy  loadings 
with  environmental  simulation.  The 
were  as  follows : 

This  is 
logically 
rating 
had  to  do 
loadings 

Simulation  of 

internal  operating  environment 

.51 

Simulation  of 

external  operating  environment 

.43 

Factors  8,  9,  and  10  showed  appreciable  loadings  by  only 
a  single  rated  characteristic  so  no  attempt  at  interpretation 
was  made.  It  is  interesting  to  note  that  all  of  the  trainer 
characteristics  included  in  the  rating  scale  loaded  heavily  on 
ac  least  one  common  factor  with  the  exception  of  the  last 
scale,  "Comparison  with  Similar  Trainers."  The  reasons  for 
this  unexpected  result  are  not  clear.  This  particular  scale 
was  intended  as  a  kind  of  summary  evaluation  after  all  of  the 
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other  trainer  characteristics  had  been  considered.  However, 
it  loaded  only  .22  on  Factor  1  (Recognized  Value)  and  .29 
on  Factor  5  (Operational  Readiness).  Its  loadings  on  other 
factors  were  negligible  with  the  exception  of  Factor  8  (.33) 
which  could  not  bf  defined.  Possibly  this  overall  comparison 
scale  led  to  a  complex  set  of  considerations  on  the  part  of 
the  raters  which  did  not  produce  much  reliable  variance.  Or, 
perhaps  a  major  difficulty  was  that  many  of  the  respondents, 
being  relatively  new  to  the  weapons  system  which  the  trainer 
represented,  had  insufficient  past  experience  in  comparable 
trainers.  In  either  case,  the  value  of  this  particular  scale 
seems  in  doubt. 

In  effect,  the  results  of  the  factor  analysis  were  to 
reduce  the  original  22  trainer  characteristics  included  in 
the  rating  scale  to  just  7  basic  dimensions: 

1.  Recognized  Value 

2.  Tactical  Realism 

3.  Provision  for  Evaluation 

4.  Performance  "Feel" 

5.  Operational  Readiness 

6.  Communication  Provisions 

7.  Environmental  Simulation 

If  the  rating  scale  were  to  be  revised  for  future  use, 
it  could  be  argued  that  it  need  include  only  the  7  factors 
listed  above  since,  generally  speaking,  those  are  all  the 
dimensions  that  were  discriminated  by  the  raters.  We  would 
argue  against  this  procedure  except  in  the  case  of  Factor  1, 
Recognized  Value.  It  would  seem  that  some  combining  of  the 
six  variables  loading  on  this  factor  could  occur  without 
serious  loss  of  information.  From  the  viewpoint  of  trainer 
acceptance,  the  three  variables  loading  heaviest  on  this 
factor  probably  reflect  the  most  important  considerations. 

With  respect  to  the  other  factors,  however,  it  is  doubt¬ 
ful  whether  combining  could  occur  without  serious  loss  of 
diagnostic  information.  For  example,  although  four  trainer 
characteristics  loaded  heavily  on  Factor  2,  Tactical  Realism, 
elimination  or  combining  of  any  of  those  characteristics  with 
any  others  might  result  in  obscuring  important  deficiencies 
(or  advantages)  of  a  particular  trainer. 


/ 
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SECTION  VII 

SUMMARY  AND  CONCLUSIONS 

•\ 

Factors  influencing  the  level  of  acceptance  of  16  complex 
training  devices  representing  air,  surface,  and  subsurface 
operations  were  studied  by  means  of  specially  developed  rating 
scales  and  questionnaires.  A  total  of  326  Navy  personnel, 
representing  students,  instructors,  administrators,  and  main¬ 
tenance  technicians,  participated  in  the  study. 

A  composite  criterion  of  trainer  acceptance  was  developed 
that  was  comprised  of  ratings  of  trainer  effectiveness,  judged 
adequacy  of  simulation,  volunteered  statements  concerning  the 
merits  and  shortcomings  of  each  trainer,  percent  utilization, 
and  ratio  of  hours-used  to  hours -requested .  On  the  basis  of 
this  composite  criterion,  both  highly  accepted  trainers  and 
trainers  with  acceptance  problems  were  identified. 

The  value  of  training  devices  to  the  overall  training 
process  is  widely  recognized  by  Navy  personnel.  Training  de¬ 
vices  are  particularly  well  accepted  with  respect  to  the 
opportunity  they  provide  to  supplement  operational  training, 
especially  the  training  they  provide  in  certain  procedures 
that  are  too  costly  or  too  hazardous  to  be  practiced  in  the 
operational  environment.  They  are  appreciated  for  the  effi¬ 
ciency  they  represent  with  respect  to  the  time  required  to 
achieve  skills  initially,  and  for  maintaining  acquired  skills 
through  a  program  of  repeated  use. 

However,  it  is  to  be  emphasized  that  no  trainer  is 
totally  "accepted"  or  "unaccepted."  There  is  little  doubt 
that  the  acceptance  of  training  devices  can  be  increased, 
with  the  consequence  of  both  improved  training  and  greater 
cost  effectiveness.  The  following  general  findings  of  the 
study  are  offered  with  these  objectives  in  mind. 

SIMULATION  FACTORS  THAT  INFLUENCE  ACCEPTANCE 

1.  Using  the  composite  criterion  of  acceptance  described 
earlier,  highly  accepted  trainers  were  rated  systematically 
higher  than  those  with  acceptance  problems  on  8  of  the  10 
simulation  characteristics  studied. 

2.  Highly  accepted  trainers  were  rated  as  particularly 
superior  on  simulation  of  vehicle  performance,  effects  of 
controls,  display  of  information,  and  sensor  performance. 

3.  The  trainers  as  a  group,  whether  highly  accepted  or 
not,  were  rated  below  the  considered  level  of  importance  on 
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three  simulation  characteristics:  displayed  information, 
sensor  performance,  and  target  reactions. 

4.  All  trainers  were  rated  high  on  communication  pro¬ 
cedures  but  almost  all  were  rated  low  on  communication 
problems.  However,  the  simulation  of  communication  problems 
was  not  considered  highly  important  by  the  participants. 

This  may  reflect  an  overly  restrictive  definition  of  com¬ 
munication  problems  in  the  rating  scale  used  for  the  study. 

5.  "Unaccepted"  trainers  were  judged  poorest  in  simu¬ 
lation  of  the  external  environment,  vehicle  or  equipment 
performance,  control  effects,  sensor  performance,  and  weapon 
firing  and  control. 

6.  Flight  trainers  appear  to  experience  acceptance 
difficulties,  regardless  of  their  other  merits,  if  they  are 
rated  low  on  simulation  of  vehicle  performance  and/or  control 
effects  ("feel") . 

7.  Weapons  systems  trainers  appear  to  experience  accep¬ 
tance  difficulties,  regardless  of  their  other  merits,  if  they 
are  rated  low  on  simulated  information  displays,  sensor  per¬ 
formance,  or  target  reactions. 

8.  Some  acceptance  problems  appear  to  have  developed 
because  the  simulated  performance  characteristics  of  the 
trainer  have  not  kept  pace  with  changes  to  the  operational 
equipment.  In  some  cases  the  trainer  appears  to  have  been 
programmed  on  the  basis  of  "theoretical"  performance  curves 
that  are  different  from  those  achieved  in  the  operating 
environment . 

9.  Operational  personnel  tend  to  be  very  sensitive  to 
the  specific  response  characteristics  of  the  vehicle  that 
is  simulated  by  the  trainer.  If  the  trainer  does  not  react 
in  a  fashion  very  similar  to  the  operational  equipment  it 
simulates,  it  is  considered  non- acceptab le  regardless  of 
whether  the  trainer  was  intended  to  be  a  high  fidelity  simu¬ 
lator  or  not.  Although  it  may  be  argued  that  important 
skills  can  nevertheless  be  acquired  in  such  a  trainer,  the 
attitude  of  "response-specificity"  is  a  major  obstacle  to 
acceptance . 

OTHER  TRAINER  FEATURES  THAT  INFLUENCE  ACCEPTANCE 

10.  The  highly  accepted  trainers  as  a  group  were  rated 
systematically  higher  than  trainers  with  acceptance  problems 
on  all  12  training  characteristics  and  situational  factors 
studied . 
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11.  Highly  accepted  trainers  were  rated  superior  par¬ 
ticularly  with  regard  to  problem  setup,  training  software, 
and  reliability.  Trainers  with  acceptance  problems  were 
rated  particularly  low  on  training  software,  reliability, 
comprehensiveness  of  training,  and  the  advantages  of  the 
trainer  as  opposed  to  the  use  of  operational  equipment  for 
training. 

12.  Considered  as  a  group,  the  trainers  in  the  study 
tended  to  be  rated  lowest  in  the  areas  of  training  software, 
completeness  of  performance  evaluation,  and  immediacy  of 
performance  evaluation. 

13.  The  trainers  as  a  group  were  rated  highest  with 
respect  to  the  opportunity  they  provided  to  supplement 
operational  training,  value  of  time  spent  in  the  trainer, 
and  value  of  repeated  use. 

14.  In  several  cases  trainer  acceptance  appeared  to 
have  been  greatly  reduced  by  inadequate  (or  totally  lacking) 
training  software.  In  contrast,  in  a  few  cases,  exceptionally 
good  training  software  apparently  had  greatly  enhanced  trainer 
acceptance . 

15.  Trainers  with  which  serious  maintenance  problems 
were  associated  tended  to  be  unaccepted.  There  are  four 
sources  of  maintenance  problems  frequently  mentioned: 

a.  Inadequate  technical  documentation; 

b.  Long  delays  in  obtaining  spare  parts; 

c.  Difficulty  in  maintaining  government- 
furnished  operational  components  (GFE) ; 

d.  Severe  problems  of  access,  particularly 
in  trailer-housed  trainers. 

It  is  generally  recognized  that  trainer  maintenance 
personnel  have  done  an  exceptional  job  of  coping  with  these 
problems . 

CHARACTERISTICS  OF  USERS  THAT  INFLUENCE  ACCEPTANCE 

16.  The  level  of  user  experience  in  the  area  of  opera¬ 
tions  represented  by  a  trainer  is  a  facto*  in  the  acceptance 
of  some  trainers.  In  general,  if  a  trainer  is  seen  as  having 
simulation  deficiencies,  the  more  experienced  personnel  will 
be  more  critical  than  the  less  experienced  personnel.  In 
addition,  some  inexperienced  personnel  depreciate  the  value 
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or  a  trainer  because,  in  their  view,  its  use  conflicts  with 
the  opportunity  to  Jo  the  "real"  thing. 

17.  Important  capabilities  of  some  trainers  are  not 
fully  appreciated  because  of  the  assumption  that  "the  fleet 
knows  b?st"  what  its  training  needs  are  and  how  to  use  the 
.trainer  to  meet  them.  A  related  problem  is  the  lack  of 
technical  qualifications  (and  often  inferior  rank)  of  those 
who  are  called  upon  to  critique  performance  in  the  trainer. 

18.  The  "not  invented  here"  phenomenon,  often  seen  as 
a  symptom  of  resistance  to  hardware  and  procedural  innova¬ 
tions,  appears  to  have  Its  counterpart  with  respect  to 
training  devices  in  the  form  of  a  "not  consulted  hero"  re¬ 
action.  Specific  deficiencies,  particularly  in  the  area 

of  simulation,  are  often  considered  the  result  of  a  failure 
to  consult  operational  personnel.  The  magnitude  of  any  felt 
deficiencies  may  be  exaggerated  by  this  reaction. 

A  related  problem  is  that  NTDC  and  the  various  systems 
commands  are  frequently  seen  as  unresponsive  to  perceived 
deficiencies,  the  need  for  updating  or  modernizing,  the  re¬ 
quirements  for  improved  documentation,  etc. 

CHARACTERISTICS  OF  INSTRUCTION  THAT  INFLUENCE  ACCEPTANCE 

19.  Quality  of  instruction  (including  evaluation  of 
performance)  is  a  major  problem  in  trainer  acceptance.  In 
some  cases  this  stems  from  the  fact  that  the  instructor’s 
role  is  relegated  to  training  devicemen  who  are  nor  technical 
experts  in  the  area  of  operation  represented  by  the  trainer; 
in  others,  it  is  due  to  the  assignment  of  the  instructor's 
role  to  operational  personnel  who  are  not  fully  knowledgeable 
concerning  the  trainer's  capabilities  and/or  the  learning 
process . 

20.  The  absence  of  standard  syllabi  and  graded  series 
of  problems  is  ar.  obstacle  to  effective  use  of  some  trainers 
and  therefore  to  acceptance. 

21.  A  problem  in  some  trainers  is  that  the  instruct' 
simply  has  too  many  stations  to  observe  and  cannot  possib  / 
evaluate  the  performance  of  all  students  adequately.  This 
has  a  negative  influence  on  acceptance. 

22.  Many  personnel  responsible-  ior  trainer  operation 
feel  out-of-touch  with  what  operational  personnel  feel  to 
be  their  training  needs. 

23.  There  appear  to  be  no  standard  qualifications  for 
instructor  personnel  for  most  trainers.  In  some  cases. 
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personnel  who  are  trained  primarily  in  maintenance  serve  as 
instructors.  .There  appears  to  be  no  well-defined  program 
for  trainingin  device  operation. 

24.  Training  devicemen  have,  in  many  instances,  done 
a  remarkable  job  of  self-instruction  in  learning  how  to 
operate  the  trainer.  However,  serious  problems  can  occur 
when  they  are  transferred  and  their  responsibilities  assumed 
by  less  experienced  personnel.  ThxS  can  produce  acceptance 
problems  for  appreciable  periods  of  time. 

25.  Unrealistic  programming  of  problems  (for  example, 
illogically  correlated  malfunctions  or  the  failure  to  pro¬ 
gram  sequentially  related  ones)  has  an  adverse  effect  on 
trainer  acceptance. 

PATTERNS  OF  USE  THAT  AFFECT  TRAINER  ACCEPTANCE 

26.  Acceptance  (or  rejection)  of  a  trainer  is  fre¬ 
quently  in  conflict  with  the  original  design  objectives  of 
the  trainer.  Most  often,  the  trainer  is  used  in  a  manner 
that  is  considerably  less  sophisticated  than  its  designed 
capabilities.  However,  such  trainers  are  often  well 
"accepted."  In  other  cases,  particularly  with  flight  train¬ 
ers,  greater  sophistication  is  expected  by  the  users  than 
was  apparently  intended  by  the  designers.  Such  trainers 
experience  considerable  rejection  by  a  significant  proportion 
of  users. 

27.  When  some  characteristics  of  the  problems  presented 
in  trainers  (for  example,  tactical  problems)  are  "too  easy" 
as  judged  by  experienced  personnel,  the  trainer  tends  to 
regress  towards  the  status  of  an  elementary  procedures 
trainer,  regardless  of  the  fact  that  it  may  have  some  very 
sophisticated  capabilities. 


28,  The  full  capabilities  o 
appreciated  by  a  substantial  prop 
tial  users)  because  of  the  lack  o 
personnel  of  what  the  trainer  can 
highly  motivated  administrative  p 
promoted  the  trainer's  use.  In  o 
this  responsibility.  In  many  cas 
tion  link  that  should  relate  flee 
capabilities  is  in  need  of  consid 
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In  the  interest  of  impro/ing  the  acceptance  and  utiliza¬ 
tion  of  training  devices,  the  following  actions  are  recommended. 
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1.  The  acceptance  profiling  technique  developed  for 
this  study  should  be  applied  periodically  to  all  complex 
trainers.  Data  on  simulation,  training  characteristics, 
and  situational  factors  influencing  acceptance  should  be 
accumulated  so  that  a  body  of  knowledge  is  developed  that 
will  be  useful  for  design  decisions.  Longitudinal  studies 
should  be  conducted  to  determine  how  sensitive  the  technique 
is  to  changes  in  the  level  of  acceptance.  The  reactions  of 
Navy  personnel  to  this  survey  indicates  that  they  would 
readily  cooperate  in  such  a  program. 

2.  Effort  should  be  made  to  increase  acceptance  and 
methods  of  utilizing  complex  trainers  by  increased  invest¬ 
ments  in  training  software.  Detailed  instructional  guides, 
prepared  by  personnel  who  are  both  operationally  knowledge¬ 
able  and  sophisticated  in  training  methodology,  should  be 

a  standard  item  for  delivery  with  all  complex  trainers. 

3.  Corresponding  efforts  should  be  made  to  ameliorate 
the  acceptsmce  problems  generated  by  inadequate  technical 
documentation,  spare  parts,  and  troublesome  GFE  items  that 
adversely  influence  trainer  maintenance. 

4.  Research  should  be  conducted  into  the  general  prob¬ 
lem  of  trainer  acceptance  as  a  function  of  the  "feel"  of  the 
simulated  vehicle.  The  fundamental  issue,  of  course,  is  the 
transferability  of  responses  from  the  trainer  to  the  opera¬ 
tional  equipment.  Rightly  or  wrongly.  Navy  personnel  feel 
the  need  for  a  high  level  of  specific  response  realism  in 
operational  trainers  (i.e.,  they  feel  the  trainer  should 
react  like  a  specific  model  of  an  aircraft,  submarine,  etc.). 
Their  reactions  clearly  reflect  a  concern  for  the  possibility 
of  negative  transfer.  Probably  this  concern  is  justified 
for  some  types  of  trainers  and  not  others.  Research  should 
be  conducted  to  define  design  guidelines  in  this  area  and 

to  better  relate  them  to  the  phenomena  of  transfer. 

5.  Similarly,  research  should  be  conducted  into  the 
necessary  fidelity  of  simulation  for  tactics -related 
training  such  as  sensor  capabilities,  information  displays, 
target  behavior,  and  weapon  reactions.  The  problem  is  to 
decide  what  levels  of  simulation  are  required  for  effective 
training  of  different  members  of  the  tactical  team  for  such 
tasks  as  target  detection,  classification,  tracking,  local¬ 
ization  and  attack. 

6.  The  problem  of  developing  uniform  and  high  standards 
of  qualification  for  instructors  needs  to  be  solved,  along 
with  effective  ways  to  train  these  personnel.  The  Navy  should 
seriously  reconsider  its  traditional  policy  of  assuming  that 
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operational  personnel  can  identify  their  own  training,  short¬ 
comings  and  act  as  their  own  instructors  in  solving  them. 

7.  For  every  training  device  representing  a  major  in¬ 
vestment,  consideration  should  be  given  to  the  development 
of  a  "trainer  advocate"  wh( ,  by  virtue  of  his  related  opera¬ 
tional  experience,  knowledge  of  the  trainer's  capabilities, 
limitations,  and  design  objectives,  and  understanding  of 
good  training  practice,  can  effectively  close  the  communi¬ 
cation  gaps  between  trainer  users,  designers,  operational 
commands,  and  systems  commands. 
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APPENDIX  A 


SAMPLE  RATING  SCALES 
AND 

QUESTIONNAIRE 
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NAME  OF  RATER: 


FACTORS  IN  TRAINING  DEVICE  ACCEPTABILITY 
PART  Is  RATING  SCALES 


INTRODUCTION 

Human  Factors  Research,  Inc.,  is  under  contract  to  the 
Navy  to  determine  why  some  training  devices  are  highly- 
accepted  by  those  who  use  them,  and  others  develop  an  un¬ 
favorable  reputation.  This  can  be  a  very  serious  problem, 
since  a  trainer  typically  represents  a  large  investment  of 
money,  and  since  the  effectiveness  of  the  trainer  might  be 
Influenced  by  its  acceptability. 

One  approach  we  are  using  to  study  this  problem  is  to 
get  information  directly  from  the  users  of  several  training 
devices.  For  purposes  of  comparison,  some  of  these  trainers 
we  are  studying  are  highly  accepted,  while  others  may  not  be. 

The  data  we  wish  to  obtain  from  you  will  require  two 
stages.  First,  this  set  of  rating  scales  will  allow  you  to 
give  evaluations  of  several  important  characteristics  of 
Che  trainer  in  a  quick  and  easy  form.  Second,  there  is  a 
questionnaire  for  you  to  record  some  detailed  reactions  to 
the  training  device,  its  effectiveness,  and  its  acceptance. 

In  addition,  a  member  of  the  research  staff  may  contact  you 
for  further  detail  or  clarification. 

In  order  to  match  up  the  rating  scales,  questionnaires, 
and  interview  notes,  you  are  asked  to  put  your  name  above, 
on  the  line  which  says  "NAhE  OF  RATER."  Flease  feel  free 
to  answer  the  questions  honestly  and  frankly.  Our  report 
to  the  Navy  will  be  in  terms  of  trends  and  averages;  only 
our  company  research  staff  will  see  the  actual  rating  scales 
and  interview  notes. 
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(Sample  page  from  rating  scale  illustrating  format) 


SIMULATION  OF  SENSOR  PERFORMANCE 


How  realistic  is  the  simulation  of  sensor  equipment  in  the  trainer?  Consider  such 
factors  as  the  physical  appearance  of  objects  in  the  environment  and  the  environ¬ 
ment  itself  as  displayed  to  the  eyes  or  ears;  the  number  and  variety  of  objects 
simulated;  how  detectability  is  affected  by  the  surrounding  environment.  If  applica¬ 
ble,  consider  how  well  different  classes  of  targets  are  simulated. 


15  r  (  ] 
14--  [  ] 

13  4*  [  ) 


12  -  [  ] 
11 --  [  3 
10-  [  ] 


®  T  [  ] 


The  simulated  presentations  cannot  be  distinguished  from  the  way 
objects  or  targets  appear  in  the  operating  environment. 


The  simulated  presentations  are  somewhat  different  from  operational 
equipment,  but  training  is  not  seriously  limited. 


The  simulated  presentations  are  sufficiently  different  from  operational 
equipment  that  some  functions  cannot  be  trained  effectively,  or  train¬ 
ing  is  generally j impaired. 


The  simulated  presentations  are  so  different  from  operational  equip¬ 
ment  that  what  is  learned  in  the  trainer  could  have  a  negative  effect 
on  operational  performance. 


(  J  NOT  APPLICABLE 


(  3  INSUFFICIENT  KNOWLEDGE  OR  EXPERIENCE 
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(NOTE:  The  questions  on  the  following  pages  were  asked  of 
all  qualified  respondents.  In  the"  interest  of  saving  space, 
the  format  of  the  questionnaire  has  not  been  reproduced.) 


FACTORS  IN  TRAINING  DEVICE  ACCEPTABILITY 
PART  II:  QUESTIONNAIRE  (TRAINEES) 


Please  fill  out  PART  I:  RATING  SCALES,  if  you  have 
not  already  done  so,  before  going  on  to  this  question¬ 
naire. 

The  form  of  this  questionnaire  is  intended  to 
encourage  somewhat  detailed  responses  to  the  questions, 
without  taking  a  great  deal  of  time.  Try  to  give  enough 
detail  so  that  the  research  staff  can  clearly  understand 
your  answers. 

If  you  wish  to  make  comments  about  the  trainer  or 
about  your  reactions  to  it,  but  these  comments  do  not  fit 
any  of  the  questions  specifically  asked,  please  write 
them  on  the  last  page  of  this  questionnaire.  We  welcome 
such  comments,  since  they  can  point  to  unexpected  factors 
in  trainer  acceptance  and  effectiveness. 

Please  notice  that’ there  are  questions  on  both  sides 
of  each  page.  Experience  indicates  that  the  questionnaire 
can  be  completed  in  about  an  hour. 


151 


NAVTRADEVCEN  70-C-0276-1 


INTERVIEW  SCHEDULE 

(ADMINISTRATIVE  AND  INSTRUCTOR  PERSONNEL) 


Name  _ 

Rank/Rate 


Length  of  Service: 


Present  Billet 


How  long? 


Yrs . 


Yrs 


Mos. 


Mos . 


Educational  1  eve! --hi ghest  grade  in  school: 

7  8  9  10  11  College  degree?  [  ]  Yes  [  ]  No 

12  13  14  15  16  Graduate  study?  _  (Years) 

Navy  schools  related  to  present  billet:  _ 


Area  of  specialization  in  the  Navy:. 


Relation  to  this  training  device: 

[  ]  Administration  [  ]  Instructor  [  ]  Trainee 

[  ]  Maintenance  [  ]  Other 

Details:  _ 

How  long  have  you  been  involved  with  this  device? 

Yrs.  Mos. 
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1.  Are  the  personnel  scheduled  into  this  trainer  all  essen¬ 
tially  the  same  in  background,  experience,  previous 
training,  and  so  forth,  or  are  there  important  differences 
among  the  trainees? 

2.  What  are  the  differences? 

3.  In  what  ways  is  the  trainer  used  differently  for  different 
types  of  trainees? 

4.  Is  student  use  of  this  trainer  voluntary  or  is  it  scheduled 
by  authority? 

5.  Do  you  think  the  amount  of  time  scheduled  for  individual 
students  or  teams  is  too  much,  too  little,  or  about  right? 

6.  Why  do  you  feel  this  way? 

7.  How  often  do  students  use  it  or  try  to  use  it? 

8.  Is  this  often  enough  to  satisfy  student  training  needs? 

9a.  Why  don't  students  use  the  trainer  enough  to  satisfy  their 
training  needs? 

9b.  What  particular  training  needs  do  you  feel  this  device 
satisfies? 

10.  In  general,  when  students  come  into  this  trainer,  are  they 
adequately  prepared  to  get  maximum  benefit  from  using  the 
trainer? 

11.  For  personnel  who  would  benefit  by  using  this  trainer, 
are  the  administrative  procedures  for  gaining  access  to 
it  easy? 

i 

12.  What  are  the  administrative  obstacles? 

13.  Do  you  know  of  any  administrative  obstacles  at  all? 

14.  Is  the  number  of  trainers  or  their  physical  location  a 
factor  in  utilization? 

f 

15.  Do  problems  in  getting  a  team  together  influence  its  use 
or  its  effectiveness? 

16.  Approximately  what  is  the  utilization  rate  for  this  trainer 

l7a.  What  would  you  say  is  the  biggest  hindrance  to  maximum 
uti  1  i  zati  on? 

17b.  What  other  factors  interfere  with  maximum  utilization? 
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18.  In  general  are  the  instructors  who  operate  the  trainer 
well  qualified  to  operate  it? 

19.  Why  do  you  say  that? 

20.  Are  they  well  qualified  in  the  skills  which  the  trainer 
trai ns? 

21.  Why  do  you  say  that? 

22.  Do  you  think  it's  important  that  they  be  skilled  in  these 
skills? 

23.  In  general  do  you  think  the  instructors  feel  that  the 
trainer  ioes  an  adequate  job  of  training? 

24.  Do  you  know  why  they  feel  that  way? 

25.  Are  the  instructors  well  qualified  to  evaluate  student 
performance  in  the  trainer? 

26.  Why  do  you  say  that? 

27.  On  what  kind  of  data  is  the  evaluation  based;  for  instance, 
does  it  depend  on  the  instructors'  memory,  informal  notes, 
a  checklist,  a  computer  read-out,  or  what? 

27a.  What  kind  of  information  does  the  trainee  get,  and  at  what 
points  during  his  training  is  he  evaluated?  Do  the  evalua¬ 
tions  pertain  to  team  performance  or  individual  performance? 

27b.  Is  the  evaluation  of  performance  in  the  trainer  reported  to 
anyone  other  than  the  students  being  trained?  If  so,  to 
whom  is  the  report  made,  and  what  is  done  with  the  in¬ 
formation? 

28.  How  could  the  evaluation  be  improved  to  be  more  useful  to 
the  students? 

29.  We're  trying  to  find  out  what  people  are  likely  to  hear 
about  this  trainer  before  they  use  it.  What  favorable  or 
unfavorable  comments  did  you  hear  before  you  had  direct 
experience  with  it? 

30.  From  whom,  and  where? 

31.  Based  on  your  experience  with  the  trainer,  what  would  you 
be  likely  to  say  to  someone  else  about  it,  both  pro  and 
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As  far  as  you  know,  does  the  trainer  have  any  kind  of  a 
reputation  with  regard  to  maintenance  or  reliability? 

For  instance,  is  it  regarded  as  a  generally  reliable  or 
generally  unreliable  machine?  I'd  like  to  know  if  the 
trainer  has  a  reputation  in  this  respect,  even  though  it 
might  be  different  from  your  own  opinion. 


What  reputation  does  it  have  with  regard  to  maintenance 
or  reliability? 


With  whom  does  it  have  this  reputation? 


To  your  knowledge,  has  it  ever  been  necessary  to  cancel 
or  delay  scheduled  training  due  to  equipment  casualties? 


How  many  times  has  this  happened  to  you? 
How  long  were  these  delays? 


Are  there  any  particular  maintenance  problems  that  you're 
aware  of  from  your  own  personal  knowledge? 


Are  there  any  particular  maintenance  problems  that  you've 
become  aware  of  through  scuttlebutt? 


Do  you  know  of  any  person  or  group  of  people  at  the  present 
time  who  are  actively  promoting  the  use  of  this  trainer? 


Who  are  they? 

What  are  they  doing  to  promote  the  trainer? 


What  positive  or  negative  effects  would  you  say  these 
efforts  are  having  on  the  use  of  the  trainer  or  on  peoples' 
attitudes  or  knowledge  about  the  trainer? 


To  your  knowledge,  are  there  any  people  now,  or  have  there 
been  any  in  the  past,  with  an  unfavorable  opinion  of  the 
trainer? 


Who  are  they? 

Why  do  they  feel  this  way? 


To  what  extent  and  in  what  ways  has  this  negative  opinion 
spread  to  other  people? 


During  the  development  of  this  trainer,  did  the  designers 
obtain  or  try  to  obtain  information  from  the  users  of 
similar  devices  (either  instructors,  trainees,  or  other 
Navy  personnel)  to  assist  them  in  making  design  decisions? 
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49.  Considering  all  the  characteristics  of  this  trainer,  the 
various  features  that  were  designed  into  it,  the  way  it 
operates,  etc.,  would  you  guess  that  users  of  similar 
devices  were  consulted  to  assist  in  making  design  decisions? 

50a.  With  regard  to  the  development  of  this  trainer,  do  you 
know  where  and  how  the  requirement  for  it  originated? 

50b.  What  are  the  training  purposes  for  which  this  trainer  was 
originally  designed?  I'd  like  to  know  what  the  designers' 
original  purposes  were,  even  if  the  trainer  is  being  used 
differently  now. 

5Qc«  Considering  the  training  needs  of  the  trainees,  what 

limitations  are  there  in  what  this  device  was  designed 
to  do? 

51.  Would  you  say  that  the  present  uses  of  the  trainer  are 
exactly  the  same  as  what  you've  said  the  designers  in¬ 
tended,  somewhat  different,  or  very  different? 

52.  Are  there  some  skills  that  cannot  be  trained  in  relation 
to  the  training  veauirement  it  was  supposed  to  fulfill? 

53.  Is  the  trainer  being  used  to  train  skills  other  than  those 
originally  Intended  by  the  designers? 

54a.  Could  the  trainer  be  used  effectively  to  train  skills  other 
than  those  for  which  it's  presently  being  used? 

54b.  Should  it  be  used  in  these  other  ways,  either  instead  of 
the  way  it's  being  presently  used,  or  in  addition  to  its 
present  uses? 

55.  Consider  the  present  uses  of  the  trainer.  Could  the  same 
training  be  accomplished  with  a  less  complex  device? 

56.  To  your  knowledge,  have  any  hardware  modifications  been 
made,  either  official  or  unofficial,  since  the  installa¬ 
tion  of  this  trainer? 

57.  What  modifications  have  baen  made? 

53.  By  what  agency  or  group  were  they  initiated? 

5$,  Have  these  modifications  affected  the  trai ni ng  capabilities 
of  the  trainer? 

60.  Have  these  modifications  allowed  more  functions  to  be 
trained,  restri cted  the  original  range  of  functions,  or 
had  some  other  effect? 
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61.  In  what  specific  ways  have  the  training  functions  of  the 
device  been  changed  by  the  modifications? 

62.  Would  you  judge  the  overall  effect  of  the  modifications 
to  be  good  or  bad,  as  far  as  training  effectiveness  is 
concerned? 

63.  What  other  effects  have  these  modifications  had,  for 
instance,  on  reliability  and  maintenance,  problem  set-up, 
and  operation? 

64.  Are  there  any  channels  of  communication  that  you  can  use 
to  make  suggestions  for  improving  this  trainer? 

65.  What  are  they? 

66.  Does  anything  effective  happen  if  these  channels  of  com¬ 
munication  are  used?  In  other  words,  do  you  feel  anything 
is  done  about  legitimate  complaints  and  suggestions? 

67.  Have  you  ever  registered  any  such  complaints  or  suggestions? 

68.  Do  you  ever  feel  frustrated  about  the  way  suggestions  or  com¬ 
plaints  are  handled? 

69.  What  would  you  like  to  see  in  the  way  of  new  lines  of 
communication  or  improvements  in  existing  lines  of  com¬ 
munication? 

70.  Have  you  had  experience  with  other  trainers  in  this  area 
of  operations?  Which  ones? 

71.  On  the  whole,  how  worthwhile  do  you  think  trainers  of  this 
type  are?  Why? 

72.  Overall,  in  terms  of  actually  accomplishing  effective 
training,  what  would  you  say  are  the  most  serious  defi¬ 
ciencies  of  trainers  of  this  type?  Please  start  with 
the  deficiency  you  think  is  most  serious,  and  give  me 
some  indication  of  why  you  feel  that  way. 

73.  If  you  were  in  a  position  to  suggest  design  changes  for  a 
new  model  of  this  trainer,  what  design  changes  would  you 
suggest  to  improve  training  effectiveness? 

74.  What  design  changes  would  you  suggest  to  make  the  trainer 
more  cost-effective? 

75.  What  design  changes  would  you  suggest  to  make  it  easier  to 
provide  students  with  more  complete  or  more  meaningful 
evaluations? 
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76,  In  order  to  provide  better  evaluations  to  the  students, 
what  kind  of  information  would  you  like  the  trainer  to 
provide  you  with? 

77.  This  is  the  end  of  the  interview,  but  before  you  go  I'd 
like  to  give  you  a  chance  to  make  any  comments  you'd 
like  to  about  the  trainer.  Perhaps  we  didn't  cover  some¬ 
thing  that  you  consider  important,  or  perhaps  something 
we  talked  about  needs  further  explanation.  Do  you  have 
any  comments  you  want  to  make? 


